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PREFACE 


This book is intended to meet the need for a concise textbook on 
learning, It is intended to be short enough to be used for a one-quarter 
course in learning and to permit the instructor to embellish where he will 
without having to spend all his time covering the text. This brevity 
has been achieved mainly by means of exclusion, The reader will find 
nothing here on human learning, on the developmental aspects of 
learning, or on the physiology of learning. All these areas have much to 
contribute to our total understanding of how behavior depends upon 
experience, but they can scarcely be more than mentioned in the first 
course in learning. So the present text is limited to the treatment of animal 
learning. The book emphasizes the theoretical issues in learning rather 
than the data of learning experiments, Again, the instructor is free 
to cite his own favorite studies to illustrate the phenomena of learning. 

The book is in two parts. The first seven chapters describe the thinking 
of the classical theorists, starting with Thorndike and Pavloy and ending 
with Skinner, In this part of the book I present the classical heritage 
upon which all modern developments are necessarily based. In the 
second part, the last three chapters, I describe some contemporary 
developments. Some of these developments are based upon recently 
discovered phenomena, whereas others are derived from new or newly 
activated theoretical ideas. It should be noted that I am not theoretically 
neutral, and in the second part of the book I actively advocate two 
ideas that are becoming increasingly popular. One is that a given 
animal's ability to learn particular things must be viewed within a broad 
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evolutionary context. The second is that learning consists not in the 
acquisition of an S-R association but in the assimilation and storage of 
information about the environment. 

I would like to thank a number of authors and publishers for permission 
to reproduce their work, Andrea Stockton for her art work, and several 
colleagues whose comments on the manuscript were most helpful. I 
would particularly like to thank several classes of students, from whom 
I learned how to organize this material, and the N 


Foundation, whose support over a number of years h 
possible, 


ational Science 
as made it all 


Seattle, Washington 
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LEARNING THEORY 


PART 


THE 


CLASSICAL 
HERITAGE 


Although a great number of people have contributed in one way or an- 
other to our current understanding of learning, there are a mere half-dozen 
or so who have made the big contribution of organizing their ideas into 
unified patterns that can be called “theories.” That these individuals have 
had the vision and courage to undertake such syntheses, often with very 


inadequate data, is reason enough to give them honored places in the anals 


of psychology and reason enough for the student of psychology to know who 
they were and what they were trying to do. Today's student should also 
understand the classical theorists because their conflicting stands on im- 
portant theoretical issues illuminate and clarify these issues. The ultimate 
resolution of these key theoretical issues will be, we hope, facilitated by a 
better understanding of how earlier theorists approached them. But perhaps 
the most important reason for understanding the classical heritage of ow 
learning theorists is that the different theorists started from different basic 
assumptions and had different views of behavior; by understanding how the 
classical theorists attempted to cope with the problems as they saw them we 
may be better able to solve the problems as we see them, 


CHAPTER 
“| 


THORNDIKE 


Edward L. Thorndike (1874-1949) was one of the first American psy- 
chologists. He was certainly the first to do learning experiments with animals, 
His doctoral dissertation, published in 1898, included the famous puzzle-box 
experiment, Thorndike’s entire professional career after 1899 was spent at 
Teacher's College, Columbia University. There he studied animal and 
human learning, the educational process, and the nature of the English 
language. Thorndike was enormously productive; in terms of the number 
of books and papers published, he was the most prolific psychologist 
ever. But, more than being a hard worker, Thorndike was a great 


innovator. 
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A great physicist once said, “In science you make progress by standing on 
the shoulders of your predecessors.” The experimental psychologist David 
Zeaman once observed that the same principle applies in psychology, except 
that “in psychology you make progress by stepping in the faces of your 
predecessors.” Zeaman may have overstated the point somewhat, but there 
is some truth in his observation. Theoretical advances in psychology have 
tended to be made at the expense of earlier theories of how things are. Each 
of the major learning theorists offered his ideas largely in defiance of 
earlier conceptions of learning, and his ideas and theories have then in turn 
been overturned and replaced by the work of later theorists. This ferment 
in the analysis of learning still continues today, so that we may expect that 
our current conceptions, procedures, and hypotheses will in time be replaced 
by sounder ideas, more sophisticated procedures, and better theories. 

The learning theorist is basically a skeptic. He looks about and observes 
that his culture, his tradition, and the prevailing theoretical ideas all tell 
him one thing. But he asks: “What if it is not like that? Suppose this or 
that assumption is false and something else is true.” He then proceeds to 
examine the implications of his new assumption. For example, one assump- 
tion that has been part of Western thought for the last 2500 years is that 
there is a profound difference between man and beast.(Man is said to have 
free will, and his behavior is said to be governed by his intellect and his 
reason. By contrast, animals are supposed to be little machines, simple 
automatons. They have reflexes and some instincts but no reason and no 
free will. This fundamental gulf between man and animal is still accepted 
by a large part of our culture. But a little more than a century ago Charles 
Darwin asked: What if it is not like that? What if there is continuity be- 
tween man and the animals? What if man just has a little more of some- 
thing than the animals have—if the difference is not in kind but in quantity? 
Man, Darwin suggested, may simply have specialized during evolution in 
the development of his intellect, precisely as other 
in the development of wings or 


animals have specialized 
in the development of a respiratory system 
that works under water. Darwin's theor 
widely accepted at first—indeed it was highly controversial, and many scien- 
tists could not accept the new idea but clung instead to the traditional 
notion of a gulf between man and beast. But with time the new idea has 
come to prevail, and Thorndike had a gre: 
this acceptance. 


y of evolution was, of course, not 


it deal to do with bringing about 


Conceptions of Learning before Thorndike 


When Thorndike came on the scene, there had already been a few men 
who had attempted to close the theoretical g: 


ap between man and animal. 
For example, Romanes (1882) had argued 


that animals are really quite 
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intelligent, in some instances showing intelligence comparable to that of 
man. His data, however, were not systematically obtained; they consisted 
anecdotal accounts of remarkable exploits by pets and other spe- 
Such arguments were interesting but had little impact. The 


e is a great gulf between man and animal. 


largely of 
cial animals. 


prevailing view was still that ther 
another very old tradition, the tradition of interpreting human 


There was 
losophers from the time of John 


learning in terms of associationism. Phi 
Locke had argued that man learns through experience and that learning 
consists of the association of ideas. For example, if a man sees the color 
white, he is inclined to think of the color black. The reason is that black 
and white sensations have been associated in his experience many times 
before, so that his mind contains the association of the respective ideas. 
Thinking of white makes him think of black; seeing white makes him think 
of black. This kind of explanation was obviously applicable only to people. 
Only people, it was assumed, can have ideas, so it 1s only people who can 
profit from experience in this manner. By contrast, animals were assumed 
to be devoid of intelligence, incapable of associating ideas. They were 
assumed to be governed by reflexes, by instincts—to be, in effect, simply 
machines, This conception of a basic difference between man and animal 
was the prevailing conception at the turn of the century and is still charac- 
teristic of the layman's thinking today. : ; 

The fundamental dichotomy between man and animal embodies the 
distinction between intelligence and instinct, or between mind and matter, 
and it probably reflects a very basic predisposition of the human mind to 


g i a eV at he observes either 4 sntalistic 
accept as an explanation of the events that h her a mentalistic 


Or a mechanistic interpretation. The behavior of man had always been inter- 
preted in terms of events in his mind. Other kinds of physical systems were 
interpreted in terms of mechanical properties; the behavior of, for example, 
rocks, billiard balls, or cannon balls was always explained in terms at 
physical events. But this distinction between the mental and the mechanical 
is purely arbitrary and traditional. Primitive peoples characteristically believe 
that physical phenomena like the stars and the hans as well as the living 
things around them, are animated by spirits: or they explain the behavior 
erms of the moods of their gods. It is thus perfectly 
ıl system in which some kind of spiritualism 
ature. This kind of approach is not limited 
Aristotle built a magnificent philosophical system in 
all things. living and inanimate alike, was explained 
iples. Even in the twentieth century Henri 
ne matter is characterized by a vital force, 


Of natural events in t 
Possible to have a philosophic: 
or mentalism pervades all ofn 
to primitive man. 
which the behavior of 
on the basis of spiritual princ 
Bergson proposed that all livir 


that is, by a sort of spiritual principle. ; ; 
By contrast, it is also possible to explain, at least in principle, the most 


mentalistic of events, the mind of man itself, in terms of mechanical princi- 
ples. Such an explanation was first attempted by the early atomistic philoso- 
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pher Democritus. In more recent times, particularly since the end of the 
nineteenth century, it has become increasingly fashionable to extrapolate 
from the physical and biological sciences to explain the activities of the 
animals and even of man in mechanistic terms—in terms of the nervous 
system or the biochemistry of the body. Although this kind of approach has 
always had some appeal, by the end of the nineteenth century most psy- 
chologists were content with a dualistic philosophy; they assumed a world 
of the mind to explain man and his doings and a physical world to explain 
mechanical systems, as well as living things excluding man, 

But, as psychology has developed in the twentieth century, one of its most 
important achievements has been the reconciliation of the inconsistencies 
in this dualism. What has happened is that psychology has gradually devel- 
oped a third kind of explanatory system, one that is concerned primarily 
with the laws of behavior, without making any commitments to whether the 
underlying causes of behavior are mentalistic or mechanistic. Behavior has 
its own laws, and to a large extent these are the laws of learning. In recent 
times Skinner has made this point of view rather well known, But the tradi- 
tional dualism has been difficult to dispel. 
theorists were not entirely consistent; some 
whereas others could be accused of being n 
difficult even for psychologists to see that there can be other kinds of ex- 
planatory models, models that are neither mentalistic nor mechanistic but 
are purely psychological in character. Let us see how Thorndike first 
attempted to deal with this problem. 


One difficulty is that the earlier 
of them were frankly mechanistic, 
lentalistic. Apparently it has been 


The Puzzle-Box Experiment 


‘Thorndike’s famous puzzle-box experiment w 
doctoral dissertation research begun at Harvard University and completed 
at Columbia. A hungry cat was put in the box, and outside was a dish of 
food. In order to get out and eat the food, the cat had to work a mechanical 
contrivance attached to the door. In one instance, for example, a string was 


as conducted as part of his 


€ increasingly 
The performance of several individual an 
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that there was considerable trial by trial variability, but also that there was 
steady progress. We can visualize a smooth “learning curve” indicating that 
the expected performance of a cat on a given trial gradually improves with 
the number of prior trials. 
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Trials 


Fig. 11 Trial by trial learning curves of five cats 
required to pull a loop of string to escape from a 
puzzle box. The cats were trained about 10 trials a 


day. (From Thorndike, 1898.) 


That is all there was to the basic experiment, but Thorndike ran a 
number of variations of the procedure, requiring different responses, and 
looking at other kinds of animals. We will have occasion to refer to some 
of his other research, but for the moment let us turn to Thorndike’s theo- 
retical account of the cat in the puzzle box. He asked what it is that is 
learned in this kind of situation. Granted that something is associated with 
something, precisely what psychological elements are associated to produce 
the improvement in behavior? He noted that there were a number of logical 
Possibilities, according to the logic of that era. Learning in people was 
assumed to consist of the association of ideas. Was it possible, for example, 
that for the cat the idea of pulling the string became associated with the 
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idea of the puzzle box and the food outside? Another possibility re Al 
idea of pulling the string became associated directly with the stimulus si “el 
tion itself, so that when in the box the cat conceived the idea of pulling the 
string. Thorndike dismissed both these pos 


ibilities on the following grounds. 
He argued that, were the cat suddenly to “get the idea,” the learning ae 
would show more or less random behavior up to the point at which the 
animal grasped the idea and that there would then be a sharp ee NT A 
in behavior and merely random fluctuation after that point (sce Figure Pa» 
We would say of human behavior that the person gets insight into on 
solution of the problem. He would suddenly understand the principle a 
the door and the latch and how to get out. But Thorndike emphasized that 
none of his cats showed this insight type of learning curve. In every instance 
the data suggested a relatively slow, 
in performance over successive trials, 


radual, and continuous improvement 
Thorndike therefore reached the very 
important conclusion that the learning of his cats probably consisted of the 
automatic formation of an association between the stimulus situation and 
the correct response. The animal learns not a connection between ideas but 
rather some kind of direct connection between the stimulus and the response. 


Time 


Trials 


Fig. 1.2 Trial by trial performance that might be 
expected from a cat ina puzzle box if the learning 


consisted entirely of “insight” into the correct man- 
ner of solution. 


This was a new idea. Learning in people h 
an intellectual kind of process, involvin 
awareness of logical relations, 
believed to be synonymous w 


ad always been assumed to be 
g ideas, appreciation of experience, 
and so on. These mental activities were 


ith intelligence. The other side of the dualistic 
picture was the reflexive or instinctive behavior patterns of animals, which 


were assumed to be fixed. But Thorndike was proposing to throw out this 
dichotomy and abolish this difference between man and animal. Here his 
cats were behaving intelligently, in the sense that they 


were learning to get 
out of the box; that is, they were modifying their bel 


lavior as a result of 
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experience, but the modification was accomplished by a mechanical kind of 
process, It was automatic; it did not require insight or ideas. Animals had 
been credited for some time with fixed or reflexive S-R connections, but 
Thorndike was proposing that there could be learned S-R connections. To 
put it another way, Thorndike argued that his cats were displaying intelli- 
gent, even creative, behavior in learning to get out of the box, but that 
this learning occurred without thinking and with minimal awareness on 
the cats’ part. 

By the end of the ninetee 3 
recognize a special form of S-R learning, which was said to occur in the 
context of well-practiced motor skills. A response that is initially voluntary 
becomes, with sufficient practice, automatic or habitual. The layman still 
uses the word “habit” in this sense. It was generally supposed that if a 
voluntary or conscious act were repeated often enough it would become 
attached directly to a stimulus, so that it would no longer require aware- 
ness. It could be said to occur habitually or instinctively.! In effect, Thorn- 
dike was borrowing the concept of habit and changing its meaning. Habit 
was to mean an unthinking response to a stimulus, Not only is a response 
that ultimately becomes free of consciousness a habit; so also is a learned 
been in consciousness. 


nth century, a number of writers had come to 


response that has never 


Thorndike’s Theory of Learning 


Thorndike observed that two special features of his experimental proce- 
antee learning. First, it was necessary for the cat 
1 well fed when it was put in the puzzle 
box, it might not have gotten out of the box in the first place and surely 
would not have shown any learning to get out. Mare likely, it would have 
curled up and gone to sleep. In short, Thorndike was telling us that the 
explanation of learning must also include some motivational principles, 
such as that the cat has to be hungry. Second, Thorndike told us that food 
is also necessary. When Thorndike developed his theory of learning in his 
1911 book Animal Intelligence. he put the matter quite plainly: Learning 
Occurs if and only if the response has some “effect upon the environment, 
If the effect of the response is pleasant, then learning occurs, If the effect 
of the response is unpleasant, then the behavior is weakened rather than 


dure were necessary to guar 
to be hungry. If the cat had beer 


was frequently used (for example, by such an authority as Wil- 
ate behavior, but to behavior that occurs without aware- 
his usage alive. He gave us such constructions as “Jojo 


1 The word “instinct” 
helm Wundt) to refer, not to inn 
Ness. Walt Disney helped to keep t 
knew instinctively that the boy was 
only that Jojo weit to the boy, who was on 
without having to think about it. 


on the other side of the hill.” What this means is 
the other side of the hill, and that he did so 
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strengthened. For example, if the cat is punished when it gets out of the box, 
it will quit getting out. But if the hungry cat is fed, then learning occurs, 
as Thorndike reported. This principle is Thorndike’s famous law of effect. 
In his own words: 


Of several responses made to the same situation, those which are accom- 
panied or closely followed by satisfaction to the animal will, other 
things being equal, be more firmly connected with the situation, so that, 
when it recurs, they will be more likely to recur; those which are 
accompanied or closely followed by discomfort to the animal will, other 
things being equal, have their connections with that situation weakened, 
so that, when it recurs, they will be less likely to occur. The greater the 
satisfaction or discomfort, the greater the strengthening or weakening 
of the bond. (Thorndike, 1911, p- 244) 


The law of effect maintains that learning consists of the strengthening 
of a connection between a stimulus situation and a response and that this 
connection will be strengthened (or, as we would say, reinforced) if the 
response has the effect of producing satisfaction to the animal, or weakened 
if the response has the effect of producing discomfort or an annoying state 
of affairs.2 

A lot of earlier writers had believed in and had advocated the principle 
of hedonism—not the epicurean idea that pleasure is good but the psycho- 
logical principle that people choose to do what is pleasant. Indeed this 
hedonistic principle had been an important part of British associationist 
philosophy, starting with Thomas Hobbes and Locke and running through 
to Herbert Spencer. Thorndike's law of effect provides an explanation of 
why people appear to be hedonistic: It is not that a man chooses pleasure, 
as the mentalistic tradition had maintained, but that men make those 
responses that in the past have produced pleasure because previous pleasures 
have strengthened, that is, reinforced, those particular responses. Prior 
learning produces just that behavior that is likely to maximize pleasure. 
Thus, although the Principle of hedonism offered a serviceable description 
; of behavior, Thorndike’s law of effect appeared to be more basic. Actually 
| Thorndike rarely used the words “pleasure” and “pain.” He chose instead 
| to substitute the behaviorally defined words “satisfiers” and “annoyers.” A 
satisfier was defined as a state of affairs “which the animal does nothing to 


2 The idea that satisfaction 


or dissatisfaction, or pleasure and pain 
Stronger or w 


eaker connections in the nervous syste: 
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avoid, often doing such things as attain and preserve it,” and an annoyer 


was defined as a state of affairs that the animal commonly avoids and 
abandons (Thorndike, ISL, p: 245), Thus we can see that for Thorndike 
the ultimate explanation of behavior lay in the orderliness of behavior itself, 
not in what he took to be mentalistic concepts like pleasure and pain. 
Although Thorndike was relatively careful to strip the mentalistic con- 
cepts, or at least mentalistic explanations, from his behaviorism, he was not 
as careful to keep out neurological concepts. Indeed, he seems to have been 
unable to resist the temptation to reduce the law of effect and his other 
behavioral principles to physiological events. He spoke about neurons firing 
and being connected, he spoke about traces of stimuli, and he incorporated 
Most of the hypothetical neurological machinery that was fashionable in his 
time. Perhaps at that time such speculations lent an air of legitimacy or a 
Scientific aura to a theorist’s work. Today, largely under the influence of 


Skinner and other modern behaviorists, we are much more sophisticated 
vs from both mentalistic and neuro- 
logical speculation. But it is no discredit to Thorndike that he was not able 
to be a pure behaviorist, much as he might have liked to be. The incredible 
thing was that he was able to be a behaviorist at all, to make any kind of 
Systematic contribution to this third realm, when the conventional realms of 


mentalism and mechanism were so firmly established and were, in fact, the 


about separating our behavioral lav 


only explanatory realms available at the time. 


Implications of the Theory 


e still further the mentalistic thinking that 


In the a safih 
attempt to minimiz : 
: th century{ Thorndike undertook 


Prevailed at the beginning of the twentie r : 9 
a number of experiments to show that even with human subjects learning 


1S an automatic process )that builds a direct connection between a stimulus 
and a response with minimal awareness.) A human subject can provide two 
kinds of data: improved performance over trials and a statement about what 
he is aware of. Thorndike’s purpose was to demonstrate the former in the 


absence of the latter. 
In one typical experiment, subjects were instructed to learn a number 
Concept. A series of numbers was presented, each one on a card; some num- 
bers were said to be positive instances of a number concept and some not. 
At first the subjects performed at a chance level, but over a number of trials 
the group as a whole came to perform 


the concept appeared to be learned: nae 
at about 90 percent accuracy; that is, they could characterize 90 percent of 


the stimulus numbers according to whether they were or were not illustra- 
tions of the concept, Thus the problem had clearly been solved. At this 
Point Thorndike asked his subjects to verbalize what the number concept 
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was. What was the concept that everyone appeared to have learned? The 
subjects reported a marvelous variety of hypotheses: If the last number was 
odd and was wrong then the next one is right; if the sum of the digits is 
even, then it is right; if any digit is repeated then the number is right; and 
so on. There was, in fact, no numerical concept; all numbers came from a 
collection of random numbers. However there was a small smudge in one 
corner of some cards, a fingerprint type of smudge. If the smudge was there 
then that number was correct, and if there was no smudge it was not. 
Apparently Thorndike’s subjects were responding on the basis of the smudge 
because the group as a whole had learned the concept. All subjects in this 
particular experiment denied having seen the smudge or having responded 
on the basis of it. Most of them claimed that they had figured out the 
numerical concept, but clearly all their number-playing games were ration- 
alizations, or after-the-fact fabrications. (Thorndike concluded that human 
subjects can learn to respond consistently on the basis of a particular 
stimulus yet be quite unaware of what stimulus they are responding to. It 
follows, he said, that reinforcement has an automatic effect.\It must stamp 
in the appropriate S-R connections as if it were a mechanical device, one in 
which the intellect and awareness of the subjects need not play any part. 
‘Thorndike reported a vast number of similar experiments. 

More recently, there has been renewed interest in the question of learning 
without awareness. A famous experiment conducted in the Skinnerian tradi- 
tion was reported by Greenspoon (1955): 
text of a classroom demonstration. 
the room and interview 


it was originally done in the con- 
A subject was brought to the front of 
ed. During the first several minutes, members of the 
class were instructed to record the operant rate of a certain kind of verbal 
behavior, specifically, using plural nouns. After the base line was determined, 
the experimenter started reinfor cing plural nouns. Behavior was controlled 
in Greenspoon's experiment simply by giving social approval, nodding, 
smiling, saying “mhmm, that’s right,” and so on. After just a few minutes 
on this reinforcement schedule, the subject showe 


use of the reinforced verbal form 
been established, it w 
conclusion of the 


d a sharp increase in the 
s. Once this high rate of responding had 
as extinguished by removing reinforcement. At the 
experiment, the subject was asked if he 
being manipulated—did he know 
Greenspoon claimed that his subjec 

Both Thorndike’s origin 
ditioning experiments of G 
siderable criticism. (It has b 
are doing; they 


had been aware of 
that his behavior was being changed? 
ts reported having been quite unaware. 
al experiments and the more recent verbal-con- 
reenspoon and others have 


een said that, of course 
are aware of t 
are reluctant to admit that tl 
example, subjects m 
relying on it w 


been subject to con- 
, people know what they 
he contingencies in the situation, 
ley know. In Th 
ay have noticed the smudge, 
ould have seemed tantamount 


but they 
orndike’s experiment, for 
but to rely on it or to admit 
to cheating; they therefore 
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invented the impossible numerical hypotheses to cover up. Similar criticisms 
have been directed at Greenspoon’s study. It appears that when properly 
interviewed subjects can report more awareness of what has been happening 
than was apparent when Greenspoon interviewed his subjects. Recent re- 
views of this literature (e.g., Saltz, 1971) indicate that the question is still 
unresolved. 

Thorndike discovered another phenomenon, which he believed showed 
the automatic nature of learning. He called it the “spread of effect.” In a 
typical Thorndikean experiment, he would read slowly a long list of perhaps 
100 or 200 words. The words might be household items like table, chair, 
and so on. The subject was instructed that for each item 
from one to ten, which he was supposed to guess. 
Typically Thorndike would give no feedback for most of the items, but for 
an occasional item he would say “yes, that’s right” (positive reinforcement). 
At some point in the experiment the word “table” might be read, the sub- 
ject might say “six,” and Thorndike would give:no feedback. The next item 
might be “chair,” the subject might say “four, and Thorndike would say 
“right,” and so on through the list. Then, typically, after going through the 
list once, Thorndike would give a retention test, which consisted of going 
through the list a second time to see to what extent the subject would Bive 
the same or different responses to the items. Thus, the experiment consisted 
of determining the probability that subjects would repeat the responses made 
to particular items. It is obviously an impossibly difficult learning task. 
it is long, confusing, and boring; and the sub- 
ject receives feedback for only a few of his responses. But, in spite ol these 
handicaps, some evidence of learning Is found. = a petiton rate for 
responses that had been reinforced (by Thorndike saying: might ) might be 
20 percent. Thorndike concluded (erroneously, as we shall see shortly) that, 
the repetition rate would have been 10 percent, 
for there were ten possible number responses. So a es rate of 20 
percent looks like learning; there appears to be evidence for the law of 
effect. There is not very much learning because of the difficulty of the task, 
but there is some, and it is usually statistically telnet : 

Thorndike discovered that there is an increased repetition rate not only 
for the correct response—that is, for the one that liss been reinforced—but 
also for the immediately neighboring responses. The kind of results ob- 
tained are illustrated in Figure 1:3: Thorndike argued that the ESSROMSES 
Strengthening powers of reinforcement, the gare effect of hearing 
“right,” can be seen only on the specific S-R connection to which it is 
applied but also on other neighboring S-R SQITIECHONS; both those before 
and after the reinforced one. The basic phenomenon i still somewhat con- 
troversial. Some analysts, like Hilgard & Bower (1974), contend that the 
Spread of effect, which they 
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there was a number 


The list is given only once; 


if there were no learning, 


describe as the last stronghold for a Thorn- 
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Fig. 1.3 Conceptualization of the spread-of-efject 
phenomenon. The repetition rate is not only higher 
for the reinforced response but for the responses in 
a series immediately before and after the reinforced 
response. Thorndike predicted a symmetrical gra- 


dient but the results of experiments are often 
asymmetrical. 


dikean interpretation of human learning, does not really exist. But other 
theorists, like Postman (1962), take a more optimistic view. Postman claims 
that much of Thorndike’s theory is still viable and that the spread-of-efect 
phenomenon has managed to survive a great amount of criticism at both 
the theoretical and methodological levels. 

here were certain methodological problems in much of Thorndike’s 
original experiments. One of the things that marred his research was that 
he rarely used appropriate control groups. It turns out that if human sub- 
jects are asked simply to guess numbers and given no feedback at all the 
repetition rate is not 10 percent, as Thorndike supposed, but more like 15 
percent. It seems that people do not guess numbers randomly; 
avoid repetition, to use series of numbers in fixed p 
makes the prediction of numbers gener: 
than the prediction of numbers generate 
proper base line for 
than to 10 percent, 
Thorndike believed. 


they tend to 
atterns, and so on, which 


A ists at all once the appro- 
priate controls are used to assess it (see Figure 1.3). 
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require awareness, and Thorndike inspired other 


jects did not necessarily 
and to engage in research to 


psychologists to think about this possibility 


p 


prove him either right or wrong. > 


Punishment 


All of Thorndike’s early statements of the law of effect included both a 
ive law of punishment. This sym- 


positive law of reinforcement and a negati 

metry reflected the age-old custom of using both reward and punishment to 
control behavior and the perhaps equally ancient belief in the pleasure-pain 
principle. Therefore the readers of Thorndike’s 1932 book must have been 
startled to discover that although he retained the law of reinforcement, he 
repudiated the law of punishment. The law of effect was no longer sym- 


metrical: it was said to work only one way, in the positive direction. Thorn- 
dike said that his experiments had forced him to abandon the law of pun- 
ishment, These experiments were essentially like those used to demonstrate 

e was that, whereas some of the 


the spread of effect. The chief differenc á 
edback “right” (reinforcement), others 


subjects’ responses were given the fe ‘ \ apne 
were given “no” or “that’s wrong” (punishment). Thorndike found that, 
1 effect of increasing the 


whereas positive reinforcement had the expected a 
repetition rate to approximately 20 percent, “no had the paradoxical effect 


of also increasing the repetition rate to approximately 15 or 16 percent. 
There wa clearly no weakening of the “punished” response. Even though 
it now. seems doubtful that “no” produces much response strengthening, 
because we now know that the base-line repetition rate 1s higher than 10 
Percent, the data do support Thorndike’s argument that there is no weak- 


ening of the response. {n a carefully controlled study wi a large number 
1 that telling the subject “wrong” can in fact 


of subjects, Stone (1953) found 3 A d 3 is 
Increase the repetition rate slightly. Thorndike proposed again on the basis 
of relatively little experimental evidence, that when punishment is effective 
in weakening behavior, which it surely is in some situations, 1ts effect is not 

8 4 effect, but is indirect, much more 


direct and automatic, as with a positive : f 
Specialized, and apparently dependent upon what the punishment in ques- 


tion makes the subject do. We shall have more to say about this remarkable 


conjecture in Chapter 4. 


Motivation 


of learning was the law of effect. He always 
e to it and defended it most earnestly. But, 
econdary laws of learning. One, 


Thorndike’s primary law 
attached the greatest importane 
at the same time, Thorndike had some s 
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which he called the law of exercise, stated that sometimes merely making a 
particular response in a given situation will strengthen the connection 
between them. If an S-R sequence is repeated often enough, the connection 
may become strengthened. But he emphasized that such repetition is a 
particularly inefficient way to produce learning, and in his later writings he 
minimized it altogether. He produced a number of experimental results that 
seemed to show that exercise by itself rarely produces learning. When learn- 
ing occurs through repetition, typically we can find some satisfying effect 
that is being produced, either intentionally or not. So ultimately the sec 
ondary law of exercise faded out of Thorndike's own thinking about 
learning. 

But there was another secondary law to which he attached much more 
importance. It was really a motivational principle. It is not clear what we 
should call it because Thorndike himself gave it different names from ume 
to time, but let us call it the law of readiness. The basic concept is that a 
particular state of affairs will prove to be satisfying only to the extent that 
the subject is ready for it. A simple example is that food is a satisfier (a 
reinforcer) only when the animal is hungry. A sex object is a satisfer only 
when certain hormonal and developmental requirements have been met 
The same principle applies to the aversive case. A certain state of affairs 
may be an annoyer just because of the momenta 


y state of the organism. On 
the other hand, it is clear that there are events, like electric shock, that are 
nearly always annoyers. The animal is nearly always ready to find shock 
annoying. In human applications the situation is more complex. We often 
do not know what a particular individual may be ready for at a given 
moment. Is it praise, power, recognition, or success that will satisfy him? 
Human motivation depends upon characteristics of the individual, as well 
as on his immediate circumstances. But, in any event, we must know the 
person’s state of readiness before we can know whether we can reinforce 
him in a particular way. 

There is yet another complication, which Thorndike illustrated with the 
example of a person who reaches for a piece of candy. In reaching for the 
candy, a wholly different kind of act is put into readiness: eating the candy. 
The individual reaches for the candy; picks it up, which is satisfying: puts 
it in his mouth, which is satisfying; and eats it, which is also sati fying. We 
thus have a sequence of actions in which the effect of each response in turn 
puts the organism into a state of readiness for the effect of the next response. 
This is presumably how chains of behavior are built up, so that ultimately 
the individual can reach for the candy, put it in his mouth, and eat it all 
more or less as one act. (Thorndike recognized, quite properly, that there is 
an enormous latitude in the kinds of reaction sequences humans can learn. 
And, by the same token, there has to be great latitude in the kinds of 
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readiness that people can have and in the states of affairs that can be 


satisfying. 
so be applied to intellectual tasks. For 


The concept of readiness can al 
ith a pair of numbers, as in Figure 1.4, 


example, if a subject is presented w 
and instructed that they represent an arithmetic problem, he may respond 
gg e ay pees HA + “pan: sae 

13,” “42,” or even “minus 1.” Now, if the response “13” is reinforced, the 


subject will rapidly acquire the short-term readiness, or “set,” to add such 
numbers, Thus, we see that readiness itself can be learned in certain in- 
stances. Thorndike believed that a large part of human performance in- 


volves such learned readiness. 
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7 


rn that leads to different 


Fig. 14 A stimulus patte 
bject’s readiness or 
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particular way. 


“set” to respond in a 


etween Thorndike’s learning principles 
e learning system was empirically sound, 
r twenty-five years, and it was extremely 
ther hand, the motivational system was 
ontinual alteration, and it was experi- 
like the readiness interpretation of 
re so vague and the determina- 
oach never has attracted much 


There is an obvious contrast b 
and his motivational principles- Th 
it remained relatively unchanged fo 
productive experimentally. On the o 
Senerally untestable, it underwent C 
Mentally sterile. In some situations, 
the arithmetic problem, the applications a 
tion of behavior so uncertain that this app” 
attention. 

It is doubtful that Thornd 
readiness—because he kept modi 
the “susceptibility” of an S-R connec 
ened by its effect. In 1913, in his book Educa 
Was transmuted into the law of readiness. 


readi : ae 
Pee with mentalistic p 
irection and attempted to 5 
a ~ 
herve is ready to conduct—W 


ike himself was ever satisfied with the law of 
fying it. For example, in 1911 he spoke of 
tion, of how likely it is to be strength- 
tional Psychology, susceptibility 
In the attempt not to imbue 
roperties, Thorndike went too far in the other 
peculate about its neurology. He said that when 
shen its threshold is low—there will be a state 


Of readiness. If a neuron is actually called upon to conduct when in this 
State, its conduction will be satisfying. He noted that the readiness of some 
Neurons is highly modifiable, whereas the readiness of others is not. Thus, 
the nerves that conduct no) ious stimuli are never in a state of readiness. 
Viewed in these terms, motivation can be seen as the particular pattern of 
readiness that exists in the nervous system at a particular moment. By the 
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time that he published his 1935 book Thorndike had attempted to expand 
the readiness concept (though at that point he called it the “okay reaction”) 
in an attempt to deal more adequately with the apparent spontaneity and 
richness of human behavior. 


Thorndike’s Impact and Contribution 


Although Thorndike’s thinking had an enormous long-term impact, it 
was not very influential at first. Apparently most psychologists did not take 
him too seriously. We shall see in the next few chapters that his view of 
learning, particularly his hypothesis that learning depends upon the “effect” 
of the response, was almost banished from psychology. It was not until the 
1930s that Thorndike, with the help of a number of other psychologists, 
was finally able to sell the idea that learning depends upon the occurrence 
of particular events, which today we call reinforcers. It may be difficult for 
today’s psychology student to conceive of a time when learning was not 
attributed to reinforcement, but that is the Way it was for thirty years or so 
after Thorndike first proposed the idea. During the 1930s, when Thorndike 
became extremely active in the study of human learning, he also became 


the dominant figure in learning theory. In our later chapters on Hull and 
Skinner we shall see how many of their ideas flowed 


directly from Thorn- 
dike. 

Let us summarize Thorndike’s basic contributions, P: 
accomplishment was to break down the mentalistic-mech 
instinct, or man-animal dualism that had such 
thought at the turn of the century. He accomplished this feat primarily by 
emphasizing that behavior is something different from the mechanisms of 
the nervous system. ‘The basic unit used for 
longer to be an idea or a nerve cell: rather, it was to be the S-R connection.) 
It would be many years, however, before psychologists were ready to accept 
this type of approach to the study of behavior. Many of Thorndike’s early 
critics, evidently confusing what he called effect (the consequence of the 
response) with affect (pleasure and pain), rejected the law of effect out of 
hand because of its subjectivity or aura of mentalism. (It is curious that no 
one dismissed his neurological speculations on the grounds of their being 
unrealistic.) 

Thorndike also proposed that all learning involves the formation of new 
S-R connections. ‘This proposal again was a radical departure from estab- 
lished thinking. {In animals S-R connections were supposed to be fixed, 
reflexive. Learning in man was supposed to involve the association of ideas. 
Thorndike gave us a new alternative: Learning involves association, but it 
is S and R elements that are connected. This basic assumption was to playa 


erhaps(his foremost 
anistic, intelligence- 
a hold upon psychological 


describing behavior was no 
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tremendously important role in the subsequent development of learning 
theory. Indeed, it is really the S-R hypothesis that made possible all the 
developments that we are to consider in this book. 

(Thorndike also gave us the concept of reinforcement, the idea that learn- 
ing occurs when a response produces a particular kind of event, like a satis- 
fying state of affairs. Before Thorndike, learning in man had been pre- 
sumed to consist simply of the gathering of experience. 

Finally, of course, we are indebted to Thorndike for initiating l 
studies of animal learning. Earlier workers had made systematic observa- 
tions of animals and had noted the occurrence of learning, but Thorndike 
was the first to study the subject systematically)using standardized apparatus 
and standardized procedures. It should also be emphasized that( Thorndike 
introduced a variety of experimental techniques for studying human learn- 


ing and verbal behavior. \He was one of the great innovators. 


ratory 
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Thorndike’s own voluminous writings make generally poor reading and cannot 
be recommended. An important and fortunate exception is his dissertation (Thorn- 
dike, 1898), Written when he was twenty-three years old, it is spirited and even 
arrogant. It has been reprinted in the book Animal Intelligence (Thorndike, 1911), 
which contains a wealth of interesting observations in addition to the original 
puzzle-box studies and includes one of the first statements of the law of effect. 

An invaluable source on biographical information about many of the older 
Psychologists is the series History of Psychology in Autobiography, originally edited 
and published by Murchison. Thorndike’s own contribution to the series is very 
short and very uninformative (Thorndike, 1936). Joncich (1968), however, has 


written a long and loving biography that tells us much about the man, his work, 


and his academic world. i 
Postman has written two critical but generally sympathetic reviews of Thorn- 


dike’s work, focused mainly on the historical ramifications of the law of effect (Post- 
man, 1947, 1962). One of the toughest questions raised by Thorndike, and one 
dearest to his heart, was whether learning can occur without awareness. This ques- 
tion, which is still with us, has been reviewed by Dulany (1968) and Saltz (1971); 


they are generally unsympathetic to Thorndike’s position. 


CHAPTER 
2 


PAVLOV 


Ivan P. Pavlov (1849- 
enormous fame 
digestive system. 
ingenious proced 
before the turn 


amous scientist. He first won 
and distinction for his research on the physiology of the 
This research was characterized not only by a variety of 
lov's utter dedication to his work, Just 
he same time Thorndike was putting 
ntered a methodological problem that 
prove more important and more interesting than his 
Physiological research. He had discovered conditioning. From then on all 
his energies and all the resources of his labor: in Leningrad were devoted 


to conditioning. Pavlov was above all else a dedicated scientist. One st 
told of him is that, because of his distinction, 


a tory 


ory 
he was offered an increased 
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food allowance during the hard times of the Revolution but that he refused 
it until extra food was made available for his dogs. 

Let us begin Pavlov’s story where he began it, with his research on the 
digestive system of the dog. He originally discovered that, when food is 
placed in the dog's stomach, the stomach walls secrete a variety of juices that 
facilitate digestion. A series of studies showed that the amount secreted and 
the duration of secretion are functions of the kind and amount of food put 
in the stomach. To simplify the measurement of what is going on inside 
the dog, Pavlov developed an ingenious surgical technique for externalizing 
a portion of the stomach. A slice of stomach tissue was cut loose and brought 
out and attached to a hole made in the side of the body. Pavlov’s skill as a 
surgeon was demonstrated by the fact that he was able to accomplish this 
surgery without disrupting the blood supply or the nervous connections to 
the stomach. The final preparation, shown in Figure 2.1, was a dog that had, 
in effect, two stomachs. Most of the original stomach still served its primary 
function, but connected anatomically to it was a small externalized stomach, 
or “Pavlovian pouch,” the interior of which could be observed while the 
animal was digesting its meal. It was found that the secretions of the 
artificial stomach corresponded quite precisely with the secretions and other 
Activities of the original stomach. It was through this kind of preparation 
that Pavlov worked out the details of digestion, including the neural re- 

exes involved. He discovered, for example, that more is secreted and that 
Secretion is more persistent with meat than with other kinds of food. So, if 


One has an ulcer and worries about stomach acid, he should not eat meat. 
Pavloy’s research on the digestive system won for him one of the first Nobel 


Prizes (1904) in medicine. 
8.C.E.R.T., West Beng 

1S. B- Fe 
Pate. 
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The basic conclusion that Pavloy reached from this research was that 
there is an innate physiological reflex controlling the amount of gastric 
secretion in a very precise manner as a function of the amount and kind of 
food in the stomach. The same conclusion applies to salivary secretions, The 
kind and amount of saliva produced by the dog are very precisely graded 
to the kind and amount of substance put into the dog’s mouth. For exam- 
ple, food in the mouth produces a thick, chemically active sali which 
starts the digestion process, whereas a drop of acid in the mouth produces 
copious watery saliva, which serves to dilute the acid, 

But then Pavlov encountered a complication. He discovered that, if the 
esophagus was cut and externalized on the neck so that food that was 
chewed and swallowed would not pi 


into the stomach but would simply 
spill out of the animal's neck, the gastric secretion was still almost as great 
as in the normal animal when the food passed on to the stomach. He had to 
conclude that the stimulus for producing the reflexive secretion could con- 
sist not only of food in the stomach, the appropriate stimulus, but also of 
food in the mouth, an anticipatory or sis 
found that all that is necessary 


the dish from which the anim 


aling stimulus. Then Pavlov 
to produce stomach secretion is to present 
al ordinarily eats. Even sight of the attendant 
who ordinarily fed the animal would elicit secretion! The situation here 
was quite different from that involving the physiological reflex. Pavlov saw 
that there must be two kinds of reflexes. There are the physiological re- 
flexes, which are invariable and are shown by all animals of a given species; 
they are an intrinsic part of the innate organization of the nervous system. 
But then there are the other reflexes, which Pa 


avlov sometimes called “psychic 
reflexes,” and at other times called “conditioned reflexes”: they occur only 
in a given individual animal 


as a result of its particular experience. All 
dogs produce gastric secretions when food is placed in their stomachs, but 
only a dog that has had certain regular experiences will produce gastric 
secretions, or salivation, when it sees its food dish, 


Conceptual Background 


Pavlov’s conception of learning was set agi 
Thorndike’s, Learning in men was comn 
of ideas, perceptions, 
mentalistic 


ainst the same background as 
nonly interpreted as the acquisition 
logical relations, and so on, 
and not very scientific, 
was interpreted as reflexive and a 
determining physiological basis. But the discovery 
tioned, reflexes presented a dilemma. ) 
pendent upon learning, but as 


all of which are very 
Pavlov said. The behavior of 
utomatic and 


animals 
as having a completely 
of “psychic,” or condi- 
The new behavior is obvious 


i ly de- 
a physiological scientist Pavloy coul 


d not 
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interpret it in the same terms as learning in man. The new behavior had 
to be simply a new reflex. It had to be based upon physiological brain 
mechanisms like other reflexes rather than upon mental events, In other 
words, Pavlov expanded the concept of the reflex to include not only un- 
learned and genetically determined responses but also learned reactions. 

For Pavlov, all behavior of the dog is reflexive. He asked, What is the 
difference between a reflex and other kinds of behavior? We all know what 
a reflex is. If we pinch a dog's foot, he raises it. If we touch him in the eye, 
he blinks it. These responses are clearly reflexes. They always occur; they 
are invariable and are presumably due to fixed neural pathways. But how 
do such behaviors differ from the behavior commonly called instinctive? 
Consider a chicken pecking at food. Pavlov argued that in this instance, too, 
there is a definite stimulus, the sight of food, and an almost invariant 


response. Perhaps pecking is not entirely invariant, but it is certainly quite 
specific and characteristic of the species. How, then, can we distinguish 
between the eye-blinking reflex, which is obviously a reflex, and the pecking 
behavior of the chicken, which is called instinctive? Perhaps reflexes are very 
simple, typically involving single S-R elements, Wiera, instincts may in- 
volve complex integrations of behavior over time. But Ratloy argued de 
this distinction, too, is invalid. Consider, for example, the “righting reflex. 
If we throw a cat up in the air, it lands on its ist ra fhe photo. 
graphic methods of Pavlov's time were hardly ie ae rotg oe he pad 
evidently analyzed the righting reflex, and he repor ted that the cat runs 
y } ategies. It pulls in its head and feet, which 


through a fixed program of str 

ert E f I = sett 5 5i 
reduces the moment of inertia and produces twisung, then puis ontonerset 
Of feet so asita twist the body around; it maintains this posture until it is 


right side up, extends its feet and head to slow the rope a ban there it is. 
Each part of the total pattern is a simple S-R element, so that the cat needs 


5 PaE ON yens—and then that reaction until 
only ake this reaction until that hap} : 
y to make this reac itegrated series of S-R events, and, because 
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something else happens. It is an it -e oR eS 
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to be hungry, to be sexually aroused, a wie po Pavey pointed out that 
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the same argument applies to he epen i 
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basis for distinguishing between reflexes and what has commonly been 
thought of as nonreflexive behavior. 

Pavlov designated other kinds of behavior reflexes, which illustrate how 
far the ordinary meaning of “reflex” can be expanded. He describes a dog 
that was conditioning very poorly. This dog resisted confinement, was 
inattentive to stimuli, and was a difficult subject in all respects. When the 
dog was turned loose outside the laboratory, it became a normal, happy dog, 
but it reacted poorly to the confinement of the experimental situation. 
Here then, Pavloy said, we have the “freedom reflex.’ 


“self-defense reflex,” which consists of a 1 


arge number of different kinds of 
behavior—threatening, growling, biting, running away, and the like—each 


of which serves in different contexts to defend the animal. All of them taken 
together were said to constitute the self-defense reflex. One other reflex 
should be mentioned. Pavlov observed that, whenever the dog is presented 
with a novel stimulus, one that elicits no marked reflex of any other kind, 
there is still a reflexive reaction, the investigatory reflex, The dog will turn 
its head, or it will perk up its ears, or look at or attend to the new stimulus. 
The “what is it?” or investigatory reflex is, in effect, the original reflex to 
all novel stimuli. The investigatory reflex typically disappears after the 
stimulus has been presented a few times. This fading of the investigatory 


reflex, a phenomenon Originally noted and described by Pavloy, is still called 
“habituation.” 


’ Pavlov discussed a 


Pavlov's argument has been given here in some det 
haps the most explicit and most convincing statement 
usually unarticulated point of view, At one level, Pavlov’s argument can be 


interpreted simply as a justification for his continued use of the word 
“reflex” to describe all behavior, whether it be inn 


level, Pavlov was revealing a conceptual bias, In i 
is, in order to approach the problem of behavior a 


ail because it is per- 
of a very popular but 


as strictly determined 
ay that reflexive behavior is, There- 
another level of analysis, we 
pe explained on the basis of 
ected innately to a response 
bility to elicit a response (a 
ave to be caused by stimuli. 


The Classical Conditioning Experiment 


We may now describe the typical Pavlovian ex 


The Classical Conditioning Experiment 25 


in the same way. We start with the physiological reflex: Food in the mouth 
produces salivation. This reaction is invariant, unfailing. We therefore refer 
to food in the mouth as the unconditioned stimulus or US, and the reflexive 
salivation as the unconditioned response or UR.! During conditioning a 
stimulus such as a tone, which is originally neutral (some writers say “in- 
different”) in the sense that it elicits only an investigatory reflex, is paired 
with the US. After a number of such pairings, the stimulus is presented 
alone, without the US. If it elicits salivation it is called a conditioned 
stimulus, or CS, and the elicited salivation is called a conditioned response, 
or CR. 

This new reflex is supposed to arise simply as a result of pairing the CS 
with the US. Food in the mouth typically precedes salivation, rather than 
following it, as it would have to if food were to reinforce salivation. The 
“effect” of the response is of no theoretical significance. If we designate the 
biologically important stimulus (food) as S*, then we can compare the 
experimental procedures of Pavlov and Thorndike by means of the follow- 


ing schemes. 


Classical Conditioning Reinforcement Learning 


S*-R 
paired S-R-S* 
S 
In both cases the response R is assumed to become associated with or con- 
nected to the stimulus S, but the procedure for creating this association and 
the learning process assumed to produce it are quite different. There may be 
some confusion here because the US is sometimes called a reinforcer, since 
it does strengthen the CR in some sense. But the procedures are clearly 
different even when the language is confusing. There is a second point of 
difference between Pavlov and Thorndike. Thorndike had resolved the 
mentalism-mechanism question by considering behavior a third way of ap- 
proaching the question, but Pavlov took the more conservative approach 
of explaining everything by means of mechanism. For Pavlov, there simply 
was no such thing as mind. Everything depended on machinery, and the 
behavior of the animal (or of man) must necessarily reflect corresponding 
events in the nervous system. As a physiologist Pavlov was concerned with 


the nervous system, and specifically with the cerebral cortex, not with any 
id . 
lawfulness that he might find in behavior. o , 
As Pavlov came to conduct increasingly complex and sophisticated experi- 
a 


ments with his dogs, he became increasingly careful about controlling the 


is the accurate translation from the Russian because it conveys 
Pavlov's meaning that the response depends upon nothing other than the stimulus. But, 
hen the early translations into English were made it came out “uncondi- 


unfortunately, w. m ri i ri 
tioned,” and this word now implies that t i te funlesened, 
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experimental situation. He said that it is impossible to condition a dog out 
in the street because there is too much distraction. Instead of giving the 
CR, the animal will give investigatory responses to the multitude of new 
stimuli that appear haphazardly. If we want to know anything of the proper- 
ties of a particular response, we have to be careful that there are no dis- 
tracting stimuli. Even the presence of the experimenter can be detrimental 
because the experimenter is likely to be reacted to as 
Pavlov’s later experiments, the dog was conditioned in & sound-proof room 
and supported in a kind of sling, so that it could not move about. The 
experimenter isolated himself in another room, where he operated the ex- 
periment by remote control and observed the animal through a system of 
mirrors. Thus the essence of the Pavlovian experiment is that the entire 
situation is controlled by the experimenter. If some stimulus is to affect the 
animal's behavior, it can be only because 
occur at a particular time. 

The philosophy of this tightly controlled procedure contr 
that of Thorndike’'s puzzle-box procedure, for example. The latter would 
be called an operant experiment today. The subject controls most of the 
events that go on in the experimental situation, whe 
simply enforces the contingency between the response 
In what we now call a “free operant experiment” 
wait for the response to occur, whereas in 
makes it occur by presenting the US. Thus tl 
(or Skinnerian) experiments differ not only 
gencies arranged by the experimenter 
situation isolates 


a social stimulus. In 


the experimenter scheduled it to 


sts sharply with 


reas the experimenter 
and the reinforcement. 
the experimenter must 
a conditioning experiment he 
1¢ Pavlovian and Thorndikean 
in terms of the critical contin- 
but also in the extent to which the 


and controls the occurrence of events in the environment. 


Parameters of Conditioning Experiments 


Pavlov and his colleagues discovered 
conditioning. Indeed, most of what 
tioning came from his laboratory. 
consisted largely in extending Pav 
kinds of responses. 
conditioning w 


a great many of the phenomena of 
is now known about salivary condi- 
Recent developments in this country have 
lov’s basic methods to the study of other 
He evidently believed that once the laws of salivary 
: ere found they would be applicable to any other response, 
hence it made little difference what response he studied. We now know that 
this belief is fallacious. Some responses are much more conditionable than 


others, and many of the rules applying to one CR have little gener: 
another. But these problems are all recently discovered 


so in the present discussion we shall follow Pavlov a 
salivary conditioning. 
Once this decision has been made, 


ality to 
» and very difficult, 
nd consider mainly 


there is a rather limited variety of 
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eliciting stimuli or USs that can be used because the US must invariably 
and innately produce the UR. Food is one such stimulus. Pavlov typically 
used a powdered food, but his earlier research on the digestive system sug- 
gests that both the amount and the quality of secretion are functions of the 
kind of food used for the US. The precise specification of the US can thus 
be an important matter. We shall see later that these considerations become 
quite important when aversive stimuli like electric shock are used in the 
study of defensive reflexes. Pavlov also produced salivation as a UR with a 
noxious substance like weak acid. But in general we have to conclude that 
the choice of a particular UR severely limits the class of stimuli that can 
serve as a US, because the US has to be a strong stimulus, one that will 
consistently and unconditionally elicit the UR. 

Pavlov found that, in contrast to the specificity of the US, there is a great 
range of stimuli that can serve as a CS. Virtually any kind of stimulus can 
be used to signal the oncoming US: a light, a noise, a piece of paper, pre- 
sentation of a particular object or person, even the lapse of time. Thus, if 
food is given to a dog at a fixed temporal interval, say every ten minutes, 
the dog will eventually begin salivating about every ten minutes in the 
external CS. Pavlov used what may seem to us rather bizarre 
a metronome and the sound of bubbling water. 
ig these particular stimuli was easily avail- 
electronically controlled tones and fields of 


absence of any 
stimuli, like the ticking of 
Perhaps equipment for producit 
able then; today we usually use 


light. a ; 
It has been discovered that, within rather wide limits, the more intense 


the CS, the more rapidly conditioning will proceed and the larger the CR 
tends to be. At one extreme, if the CS is too weak (too near the threshold 
of detection), there may be no conditioning. Clearly, the animal has to 
detect the CS; any stimulus that ensures that the animal will give a clear 
investigatory response is likely to facilitate conditioning. At the other ex- 
treme, if the CS is too intense, so that it evokes some type of pain or fear 
reaction, rather than the investigatory reflex, then it no longer serves to 
produce conditioning, or at least not the conditioning that we were looking 
for. Instead of salivation, we obtain a defensive reaction. 

It is sometimes said that the CS must be originally neutral, that is, must 
itself evoke no particular response. This criterion is obviously too stringent. 
First of all, there may not be any really neutral stimuli, Any stimulus that 
is detectable at all is likely to invoke at least an investigatory reflex. Indeed, 
absence of such a response, conditioning 


there is reason to think that, in the 
CS must be emotionally neutral, 


will not occur. It is not even true that the 
in the sense that it evokes no appetitive or defensive behavior. It is only 


necessary that the CS be weaker or more nearly neutral than the US itself. 
If a sufficiently intense US is used, then the experimenter has considerable 
latitude in how far from neutral the CS may be. One striking illustration 
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of this plasticity of the CS is the finding reported by Spragg (1940) that a 
hypodermic syringe, which initially evoked consider: 
became, after a time, a very positive stimulus. The 
ing for it after it had been used a number of times 
tions, Pavlov had previously reported th 
as a CS to produce salivation. 


Because stronger CSs are generally more effective than weaker ones, it is 
not surprising to discover that the onset of a stimulus, like a light or a 
noise, is a more effective CS than stimulus termination, 
better than darkness or silence. 


Because the experimenter controls all the events occurring in the Pav- 
lovian situation, it is possible to manipulate systematically the time interval 
between the CS and the US in ways that are not possible in an operant 
experiment. It is often reported that an interval of about one half-second 


produces the greatest amount of conditioning. If the time interv: 


al is shorter 
than a half-second, and particularly if the interval is negative so that the 


CS follows the US, a dramatic failure of conditioning is typically found 
(see Figure 2.2). On the other hand, for intervals longer than a half-second 
the rate of conditioning decreases more modestly, so that it is possible to 
get conditioning even with fairly long CS-US intervals, provided that a suffi- 
cient number of trials are run. It should be emphasized that these data and 
these conclusions are idealized. Although there is considerable evidence for 
a half-second optimum (see, for example, Kimble, 1961), most of this evi- 
dence comes from eye-blink studies, and the evidence from studies of other 
kinds of responses Suggests that there may be very different optimum inter- 
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Fig. 2.2 How the strength of conditioning is sup- 
posed to depend upon the interstimulus interval, 
Note the hypothetical failure of conditioning under 
“backward” conditions, that is, when the US pre- 
cedes the CS. 
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vals for different responses. For example, VanDercar and Schneiderman 
(1967) conditioned rabbits with different intervals while simultaneously 
measuring eye blink (of the nictitating membrane) and heart rate. At .25 
second they found good eye-blink but no heart-rate conditioning; at 6.75 
seconds they found just the opposite pattern. We may think of conditioning 
as occurring under both conditions but being manifest only in one or the 
other response system. 

Pavlov distinguished between two kinds of procedures that he called, and 
which we still call, the “delay” and the “trace” procedures. In the delay 
procedure, the CS comes on and stays on until the US occurs. Usually the 
two stimuli overlap in time. The procedure is called delay because the onset 
of the US is delayed following the onset of the CS. In the trace procedure 
there is a momentary CS, and then at some fixed interval later the US comes 
on alone. Thus the CS and US do not overlap and are never actually 
paired. These relationships are shown in Figure 2.3. The name trace is 
given to this latter procedure because presumably the effective stimulus—the 
brain activity that really matters—is not the sensory input itself but some 
lingering trace of it. Perhaps because the trace is never as potent as the 
original stimulus, the trace procedure is not as effective asa delay procedure 
in producing conditioning with the same interstimulus interval, 

There is a methodological consideration that arises in conditioning ex- 
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Fig. 2.3 The temporal relationships between cs and 
US that define two types of delay conditioning, 
trace conditioning, and backward conditioning pro- 
cedures, 
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periments with different interstimulus intervals, Imagine an experiment in 
which there are two groups, one with an interstimulus interval of 0.3 second 
and the other with the CS and US separated by 3.0 seconds. In both cases 
we should expect to find a substantial amount of conditioning, provided 
that other parameters of the situation are appropriate. But the animals 
trained with the long interstimulus interval have ten times the amount of 
time available in which to respond to the CS before the US comes on. And 
we have ten times the amount of time in which to measure their salivation 
before the US, food in the mouth, produces a large UR that will obscure 
any CR. This is a serious methodological problem; if we are to assess the 
relative strength of conditioning in the two groups, we are obliged to use 
Some test procedure that will give the two groups an equal opportunity for 
the CR to occur, There are three Ways to do so. One is to avoid using the 
short interstimulus interval, so that all animals have ample opportunity 
to respond during conditioning. The second procedure, which was favored 
by Pavlov, is to omit the US on some trials and to record the amount of 
salivation occurring in some interval of time, like three seconds, that is the 
same for all animals, The first procedure is a little awkward because it 
limits the kinds of experiments that can be run. There is the additional 
problem that, even if the CR does occur, it may be very difficult to distin- 
guish from the UR because the two merge into each other. The second 
procedure is a little awkward because it intermixes training trials in which 
the CS and US are paired with a number of test trials in which the CS 
occurs alone. Extinction (see next section) is likely to occur during the test 
trials. A third technique is simply to discontinue pairing CS and US and to 
measure the strength of the CR during a series of extinction trials, But 
again we are limited in the kinds of experiments that we can do. 


Some Conditioning Phenomena 


Pavlov discovered extinction, named it, and was the first to study it 
extensively. The extinction procedure is defined as presenting the CS alone 
without pairing it with the US. We should distinguish carefully between 
extinction as a procedure, as something the experimenter does, 
tion as something that happens to behavior: a weakening as a result of the 
procedure. The same word is properly used in both ways (but it is not proper 
to say that the animals “extinguished” or the animals “were extinguished”), 
Pavlov observed (see Figure 2.4) that the CR may extinguish rapidly. He 
noted further that, when the spacing of the test trials was increased, the 
response extinguished somewhat more slowly than is shown in Figure 2.4, 


and extinc- 


Some Conditioning Phenomena 31 


06 
g 04 
$ 
5 
n 
0.2 
o 
(es ei eerie 
j 2 os st oe É 
Trials 


Fig. 2.4 Extinction of conditioned salivation. The 
CS was the visual presentation of food for I minute 
at 2-minute intervals, and the response measure was 
the amount of saliva secreted during CS presenta- 
tion. Unfortunately, we cannot tell from Pavlov’s 
report how often the response had been previously 
extinguished and reconditioned. (From Pavlov, 1927.) 


it is apparent that the salivary reflex is extinguished surpris- 
more rapidly than we customarily think of operant behaviors 
weakening when reinforcement is withheld. We may expect that different 
responses will be extinguished at different rates. 

There are several other phenomena that Pavlov discovered, studied, and 
gave names to which are still used in the analysis of learning. One of these 
phenomena is generalization. Consider a situation in which the CS is a 
tone of 440 hertz (cycles per second), and let us suppose that sufficient pair- 
ings have been given so that this CS elicits a strong CR. At this point let us 
suppose that we test the subject with a different tone. If the new test tone 
is fairly similar to the original CS, say 500 hertz, it will typically elicit some 
measurable CR. It will not be as strong as that obtained with the original 
440-hertz tone, but it will still have some strength. If we continue testing 
with tones of 600, 700, and 800 hertz, we should find a progressive decline 
in the strength of the response. Representative data from just such an 
experiment, reported by Siegel et al. (1968) are shown in Figure 2.5. Gen- 
ar the test stimulus is to the original training stimulus, 
the greater the response strength. Or, put the other way around, the more 
distant the test stimulus is, the greater the decrement will be. Perhaps such 
results are not surprising. If so, it is because we take generalization so much 
for granted. Generalization is actually an extremely important phenomenon. 
If it were not for generalization, learning as We know it would never be 
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Fig. 2.5 Probability of a conditioned eyelid re- 
sponse in rabbits when tested with tones of difjerent 
frequencies. The US was an electric shock to t 
cheek. The arrow indicates the frequency at whi 
each group was conditioned. (From Siegel et al., 
Generalization gradients obtained from individual 
subjects following classical conditioning, Journal of 
Experimental Psychology, 1968, 78, 171-174. Copy- 
right 1968 by the American Psychological Associa- 
tion. Reprinted by permission.) 
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traneous stimulation and inadvertent variation among stimulus conditions 
are minimized, there still has to be some variation in the total pattern of 
stimulation from one trial to the next. It may make some difference whether 
the animal is inhaling or exhaling at the moment the CS comes on. But the 
important factor is not that the stimulus situation is slightly different from 
trial to trial but that it is largely similar from trial to trial, so that the 
learning that occurs on one trial can be manifest on the subsequent trial. 

Although we tend to think of generalization as occurring along some 
easily defined dimension of a stimulus, like the frequency of a tone or the 
intensity of a light, much of the time in everyday experience generalization 
occurs from one stimulus context to another in the absence of any clearly 
defined stimulus dimensions. Consider, for example, the problem of learning 
to recognize and react in more or less the same way to a Class of stimulus 
objects, A class of objects, like all dogs, includes stimulus patterns that 
differ in color, size, shape, configuration, shagginess, pattern of movement, 
and probably a host of other even less well-defined stimulus dimensions, 
but ultimately we come to respond to any of this diverse class of objects as 
a dog. Consider further that other dogs also recognize this class as such, 
Although another dog's response to this class may be different from that of 
although the dog presumably generalizes on different dimen- 


people and 
sions from those that people use, the dog also arrives at the same function- 


ally equivalent class of dog objects. NRTA 

In one sense, the opposite of generalization is discrimination. In general- 
ization the animal responds in the same way to two similar stimuli; in 
discrimination the animal responds differently to two similar stimuli. 
Consider a procedure in which a CS+, say a tone of 400 hertz, is presented 
on half the trials and is always paired with food, whereas a CS—, say a 
tone of 500 hertz, is presented on the other half of the trials and is never 
followed by food. Under these conditions the CR begins to occur to both 
the CS+ and the CS—. But, with continued training, the response to 
CS— gradually drops out. Ultimately the animal continues to respond just 
to CS+ and does not respond to CS—. It should be apparent that, just as 
generalization is necessary for learning to be manifest in behavior, so there 
must be discrimination if an animal is to respond intelligently to the world 
about it. Just as we classify all dogs as dogs, we also discriminate between 
large and small dogs, between black ones and brown ones, between males 
and females, and so on. We might suppose that the intelligence of an 
organism could be measured by the number of different discriminations 
that it can make among a set of similar stimulus objects. The cartoon in 
Figure 2.6 shows an instance in which an individual is generalizing when 
ultimately, we trust, he will learn the appropriate discrimination between 
two classes of stimuli that are different in important ways. 
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Fig, 2.6 


Inhibition 


Pavlov argued that conditioning could involve either an active excitatory 
Process, by which a stimulus excites a particular response, or an inhibitory 
process, by which a stimulus actively inhibits a response that would other- 
wise occur, So, just as a CS+ May cause a response to occur, a different 
stimulus, a CS—, may prevent its occurrence. The simplest case is one that 
Pavlov described as external inhibition. It can be thought of roughly as 


distraction. Suppose we have a well-established CR: salivation consistently 
elicited by a tone. Suppose further th al when the tone CS 
is presented, a new distracti Sti i resented (it makes little 


difference whether the new stimulus is presented immedi 


during the regular CS). What we are li S that the occurrence of 
an investigatory reflex to the novel sti 

is no salivation. But this disruptio 
is presented on a series of trials, the CR will retur 
haps within a half-dozen trials. familiarization With, 
the novel stimulus can be accomplished ej 


itioning is be un, then when it is int s eel te 
di ning is Deg) si he rod > after conditioning, 
it will have little or no inhibitory effe $ 
was that the distracting stimulus no longer initi 
so this behavior cannot compete with or inhibit 
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A quite different kind of inhibition is what Pavlov labelled internal 
inhibition. He discovered that, if extinction is carried just to the point 
where the CR no longer occurs and the animal is then given a period of 
twenty-four hours to rest, the CR will show spontaneous recovery when 
tested again. The extinction experience does not permanently weaken the 
CR. Pavloy argued that the phenomenon of spontaneous recovery proves 
that the waning of the CR in extinction does not represent a “dying” of the 
reflex or any real weakening of the learned S-R connection. It shows, on the 
contrary, that the response is there with at least considerable strength and 
that it is actively blocked by some kind of inhibitory process. It is this hypo- 
thetical inhibition that Pavlov called internal inhibition.? 

It should be noted that, if a second extinction is begun some time after 
the first, extinction proceeds more quickly than it did the first time. The 
second spontaneous recovery is also less complete. If a series of successive 
extinctions are run, the amount of spontaneous recovery will continue to 
decline and successive extinctions will continue to go more rapidly—until 
finally a point is reached at which the CR is permanently extinguished. 

Consider again the discrimination procedure described previously, A CS+ 
is consistently paired with the US, and a CS— is consistently not paired 
with the US. As Figure 2.7 shows, responding to CS— can be expected to 
drop out. But now there is the question whether the loss of responding to 
CS— represents a fading away or decaying of that particular reflex or 
whether the reflex is still there and is being inhibited in some way. The 
critical test is to present a novel stimulus, one that would be expected to 
produce external inhibition. If such a novel stimulus is presented at the 
same time as the CS+, it produces, as we have already noted, a marked 
loss of the CR on that trial (see the left-hand part of Figure 2.7). But if the 
distracting stimulus is presented on a trial when the CS— is presented, we 
find a recovery of the CR essentially to full strength (the right-hand part 
of Figure 2.7). Pavlov called this phenomenon disinhibition, and he argued 
that it must be due to the distracting stimulus producing interference with 
an active inhibitory process. This procedure for producing inhibition, that 
is, for associating CS+ with the US and a different stimulus, CS—, with the 
absence of the US, is called the differential conditioning procedure. 


2 Note that the kind of inhibitory mechanism theoretically involved in internal inhibi- 
tion is quite different from that hypothesized to underlic external inhibition, Actually, 
both designations are somewhat inappropriate. An animal may be distracted by sudden 


internal stimuli as much as by external ones. Pavlov cites the instance of a dog's needing 
the CR through external inhibition, The 


to urinate. Here an internal stimulus suppre 
mechanism here is simply distraction or the elicitation of competing behavior. The mech- 


anism underlying internal inhibition, however, must be a specific tendency not to make 


the previously conditioned response. 
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tion procedure may all be presumed to be quite comparable. In each case 
there is one stimulus or pattern of stimuli that elicits the CR and there is 
another stimulus or pattern of stimuli that inhibits it. Presumably the same 
kind of inhibitory mechanism is at work in each case. 

A somewhat different inhibitory mechanism is suggested by the phe- 
nomenon that Pavlov called delay of inhibition. Let us suppose that we 
endeavor to establish a conditioned salivary response in a dog using a very 
long CS-US interval, perhaps one minute. We know from earlier discussion 
that such conditioning will occur slowly. But if we use an animal that has 
previously been conditioned (which Pavlov tended to do) and if we are 
patient, we can elicit conditioned salivation. Perhaps after 100 training 
trials a modest CR will occur shortly after CS onset—long before the food 
US is delivered. But if we continue training the animal another 100 or 200 
trials, the CR may change: Although it may continue to gain in amplitude, 
it may also occur later in the CS-US interval. Ultimately the animal “waits” 
until just before US presentation before it salivates. That this delayed 
response to the CS is due to some form of inhibition is shown by the fact 
that if we do a distraction experiment early in the CS, the animal will 
salivate promptly. We have thus, once again, the disinhibition demonstra- 
tion to argue that there is some active inhibition of the response early in 
the CS. 

There are two sets of phenomena that we must mention before conclud- 
ing our discussion of Pavlovian conditioning. One is interesting in that it 
shows a failure of conditioning where we might have expected it. And the 


other is a phenomenon in which conditioning seems to occur even though 
the experiment is arranged to make it impossible. Consider first the failure 
of conditioning. Suppose we have two sumuli tliat have been found in pre- 
vious experiments to be perfectly adequate as CSs, for example, a bright light 
and a weak tone. We pair both with food a number of times until the 
combination regularly elicits the CR. Then we run a series of test trials using 
either the light or the tone. Typically the bright light produces the CR but 
the weaker tone elicits nothing. It appears that the rapid formation of a 
connection between the light and the food blocks or “overshadows” any 
connection that might have become established between the tone and food, 
This overshadowing phenomenon was extensively studied by Pavlov but 
attracted little attention in this country until quite recently (Kamin, 1969). 

The second phenomenon is typically found in defensive, rather than 
appetitive, conditioning situations. It was discovered by Grether in 1938 
(Pavloy missed it somehow). Grether first tested his animals with a bell; it 
elicited little reaction. Then his animals were frightened several times by a 
brilliant flash of light (the US). The bell was not presented on these očca- 
sions; the bell and the light were never paired. But then, when the bell was 
presented alone on a test trial, it produced a strong fright reaction. Grether 
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called this phenomenon pseudoconditioning ae 
bag. : eaten . z a hee in th 
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» and the name has stuck. It is 
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this learning is best conceptualized. To keep t l 
ces ‘ ‘ . -tation ani 
proper historical perspective, we shall stress Pavlov S own interpretation al 3 
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; tane will De 
formulated. In a later chapter some contemporary interpretations will 
discussed. 


It should: be emphasized that Pavlov'’s primary concern was not to estab- 
lish a theory of learning but rather to develop techniques for 
brain. He was particularly concerned with the cere 
assumed, as did most scientists early 
Special province of the cortex. He also assumed that le 
only in an animal as high as a carnivore if the cortex 

research led him to formulate a rather specialized theory 

cortex in learning. The basic idea is that, when the US stimulates the cortex, 
it produces maximum activation at some definite point; but, because it is @ 
strong stimulus, there is a field of neural activity radiating from this point 
Similarly, the CS activates maximally a somewhat different location on the 
cortex, and it produces an area of activity Somewhat smaller than that 
produced by the US. If these two fields overlap, Pavloy assumed, the sensory 
fields stimulated by the two stimuli should become integrated in a common 
field. Specifically, a neural path should develop between the two loci points. 
The two stimuli should become functionally equivalent, 


studying the 
bral hemispheres. He 
in this century, that learning is the 
arning can occur 
is intact, and his 
of the role of the 


but because the 


*A similar type of phenomenon, Perhaps ari 
citability, is called Sensitization. If we use a stir 
defensive behavior or any particularly directed be 
we may find that this reaction is 
series of USs alone. In effect, an animal may become frightened 
has been shocked, in spite of the fact that the stimulus is neve. 
so that it is not a CS in the technical Years a goo 
sophistication has gone into the questi 
conditioning effec 
effects may reve 


ing from the same kind of general ex- 
Mulus that initially produces some mild 


1 
havior other than the 


eat š orienting reaction, 
greatly augmented SIMPly by our giving the animal a 


by a stimulus because it 


around the 
ian situation, 


The Theory of Conditioning 39 


US is the stronger of the two, producing an invariable response, the basic 
modification must be in the activity elicited by the CS. The CS produces the 
same response originally elicited by the US. Pavlov eventually elaborated 
able detail and developed it to explain 


his theory of the brain in consider 
For example, generalization was 


many of the basic facts of conditioning. 
explained as the production by similar stimuli of similar and partially over- 
lapping fields of activity. As another example, the importance of temporal 
factors in conditioning was explained in terms of how fast brain activity 
spread from the point of maximum stimulation and how fast the area of 
activity contracted afterward. 

Few psychologists today take Pavlov’s brain theory very seriously. The 
brain theory is mentioned at all only to emphasize two facts. One is that, 
in Pavlov's thinking, learning consisted not so much in the construction of a 
new S-R connection as in the learned equivalence of stimuli. The second 
point is that, for Pavlov, learning arose from the altered relation of neurons 
in the brain, especially in the sensory areas. Because Pavlov was not a 
psychologist and not particularly concerned with psychological theory, he 
did not speak very informatively about the psychological aspects of condi- 
tioning. He described these aspects of conditioning in a number of different 
ways, using a number of different analogies. For example, he conceived of 
the CS as a signal; the CS signals the US. He sometimes spoke of the CS-CR 
connection as a “signalized reflex.” At times Pavlov spoke of the CS-CR 
reflex itself as learned, whereas at other times he referred to the signal value 
of the GS or the learned equivalence of the CS and US. If the CS comes to 
acquire some of the properties of the US, including the elicitation of saliva- 
tion, then it may be that the CS signals the US, not that a new S-R con- 
nection as such has been learned. 

Although Pavlov was not very specific about the psychological nature of 
conditioning, subsequent writers have been quite specific about what they 
thought it involves. One of the most common interpretations is that con- 
ditioning consists of learning by stimulus substitution. A response that is 
originally evoked only by the US comes, through stimulus pairing, to be 
evoked by the CS. The CS literally substitutes for the US in producing the 
same response. Although this view of cl 


assical conditioning is simple and 
attractive, it is not consistent with the facts 


as they are now known. In one 
of the first successful conditioning studies reported in this country, Upton 
(1929) conditioned a fear reaction to 


auditory tones. His ultimate purpose 
was to do generalization studies in W. 


hich tones of different intensities and 
frequencies would be presented to determine how high and how gentle a 
tone different animals can hear. (This techniqu 


e still remains one of the 
most powerful tools for studying the sensory abilities of animals.) In carry- 
ing out these studies Upton found that whereas the UR to shock was 
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accelerated heart rate and respiration, the CR (the response to the tone) was 
a decrease in heart rate and an interruption in breathing. Such anomalies 
have become common in subsequent years. The CR is simply different from 
the UR in many instances. The difference appears to depend upon a num- 
ber of factors, such as the intensity of the shock, the interstimulus interval, 
and especially what kind of animal is being conditioned (see Chapter 8). 
But, in spite of evidence to the contrary, many psychologists have assumed 
that the CR is merely a small edition of the UR that occurs because the CS 
has come to substitute for the US in eliciting it. We shall see in the bent 
chapter that Watson had his own interpretation of how learning negats 
in the conditioning situation. In Chapter 8 we shall see that alternative 
interpretations of classical conditioning are still being proposed. d 
In contrast with this unsettled question of what learning in classical 
conditioning consists of, there has never been much question (until recently) 
about the procedure that produces it. Conditioning is said to occur when 
appropriate stimuli, a CS and a US, are paired. The pairing of the stimuli 
is the all-important ingredient of the Pavlovian procedure, Indeed, we have 
seen that, if the CS comes to elicit the CR without benefit of pairing with 
but pseudoconditioning instead. 
Most of this chapter has been devoted to the phenomena of conditioning 
s, most of them discovered by Pavlov himself, for 
this emphasis is that many of 
alysis of learned behavior yet have 
in terms of later theorists’ own con- 


asizing the Procedures for conditioning, 
rather than the hypothetical process of conditioning, is that Pavlov’s own 


theory of conditioning was really extremely simple. Putting aside his special 
theory of brain functioning, the general theory of conditioning can be 


these phenomena are basic to any an 


f; Conditioning is a hypothetic 


US that does elicit the response. 


3. All learning in man and beast is due to conditioning.4 
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Pavlov’s Contributions and Impact 


In his own country Pavloy attracted a vast number of other scientists to 
atory. Some were young men and students, but others were 
older, well-established scientists who dedicated themselves to his work. We 
may hope that these selfless men received the recognition due their efforts 
in Russia; in this country they are hardly more than colorful names adorn- 
ing the pages of Pavlov’s Conditioned Reflexes. We merely note in passing 
that much of Pavlov's prodigious research output was made possible by a 
ators who learned from him, to be sure, but who also 
matic program. 


work in his labor: 


corps of investig 
contributed greatly to his syste 

Although there were some early brief descriptions of Pavlov’s techniques 
(Pavlov 1906; Yerkes & Margulis, 1909), they conveyed little of the methodo- 
logical sophistication or the theoretical underpinnings that characterized 
Pavlov’s later work. A full account of his work and his colleagues’ work 
did not appear in English until 1927. But by that time the general concep- 
tion of conditioning had already gained tremendous popularity. The ele- 
gance and objectivity of conditioning procedures were advocated by Watson 
in a series of publications starting in 1916, by Smith and Guthrie in 1921, 
and by a number of other contemporary writers. 

It is remarkable that this acceptance of conditioning as a procedure and 
as a theoretical model was so widespread before there were any experimental 
reports of classical conditioning in this country and before anything was 
really known about it. There were a few scattered reports of successful 
conditioning from Watson's laboratory in 1916 and 1920 and by Mateer 
(1918) and Cason (1922), but there were more reports of failures, methodo- 
logical problems, and conceptual difficulties with conditioning." In spite of 
these problems, conditioning was accepted as both a theoretical framework 
and a practical technique for solving a variety of applied problems. Thus 
1924 saw the publication of a book by Allport on social psychology and 
another by Burnham on abnormal psychology. There was apparently no 
limit to the power or applicability of conditioning. But, as we will see in 
the next chapter, part of the success of the conditioning concept resulted 
from rather gross distortions of the theory, principles, and procedures that 


Pavlov was developing. 


5 The first conditioning experiment in this country was really done by Twitmyer. His 
doctoral dissertation (University of Pennsylvania, 1902) was a study of the knee-jerk reflex. 
He found his subjects giving the response to the ready signal before the hammer struck. 
This research attracted little attention and was abandoned; Twitmyer had been a little 


too far ahead of his time. 
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In assessing the impact Pavlov had upon the le 
lowed him, we have to distinguish 


tion he sought to make. His first ci 
brain. U 


arning theorists who fol- 
clearly among three kinds of contribu- 
oncern was to develop a theory of the 
nfortunately, his model of the brain was never very influential, and 
with time it has become less so. Even so, a few capable workers in the field 
of conditioning (like Wyrwicka, 1972) who share Pavlov's concern with 
neural mechanisms are still able to accept and defend some of his brain- 
model principles, 


r á TORS, ified 
Asa theory of learning, that is, a theory about how behavior is modifie 


through experience, classical conditioning has fared considerably better. 
We have seen that many of Pavlov’s assumptions about what learning con- 
sists of, and the kinds of mechanisms that produce it, were very widely 
accepted even before his work was well known in this country. Then, even 
as the universality of conditioning came into question—as the domain of 
conditioning shrank—there was a healthy growth of interest and activity 
connected with the phenomena of conditioning. Today there appears to 
be a resurgence of interest in classical conditioning, 
early American writers but as originally discussed by Pavlov, For example, 
his treatment of inhibition is now being follow 
ignored for many years. In short, 
markably well as a theory 
began to develop it. 


At a third level of analysis are the procedures, the phenomena, and the 
language of learning and conditioning, In this sphere Pavlov's contribution 
has been immeasurably great. Much of the terminology th 
scribe learning experiments is his. Pheno 


mena such 
tion, discrimination, and inhibition were discover 


procedures have become part of the h 

At a more abstract level, Pay 
elicited responses in animals o 
was due to the mech 
immensely provocativ 


not as distorted by 


ed up after having been 
classical conditioning ha 


survived rê- 
of learning in the 


. mere y 
Seventy years since Pavlov 


at we use to de- 
as extinction, generaliza- 
ed by him, Many of his 
animal experimentation. 


at all learning, whether of 
r of highly conceptual beh 
anisms of classical conditioning, 
e and constructive 
wrong, but it is nonetheless one of m 
its detractors and its defenders to finc 


ard core of 
lov thought th 


avior in humans, 
This has been an 
idea. We now believe it to be 
an’s great ideas. And it stirred both 
l out much more about learning. 


References for Further Reading 


There are no adequate secondary sources on Pavlovian conditioning, and the 
old primary sources are in Russian journals that are difficult to obtain, The student 
can do no better than to read the word of the scientist himself (Pavlov, 1927), in 
which he will find little data but much learned discussion of the phenomena of 
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conditioning. This book, Pavlov’s only systematic presentation, is necessary reading 
for any serious study of conditioning. A number of Pavlov’s theoretical papers have 
been translated by Gantt (Pavlov, 1928, 1941), who also provides a businesslike 
biography of Pavlov. Cuny (1966) gives a more personal view of the man. The 
classic account of the early conditioning experiments in this country is Hilgard and 
Marquis (1940). For good reviews of more recent work on selected topics in condi- 
tioning see Prokasy (1965) and Black and Prokasy (1972). 
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Watson received his degree from The University of Chicago in 1908, 
which was at that time strongly under the influence of John Dewey and 
as called functionalism. Functionalism was a 
1 the traditional belief in the im- 
d to be the device that man had 


strongly committed to what w 
strange blend of evolutionary theory and 
portance of the mind. The mind was sai 
evolved for solving his unique problems. Thus Watson was brought up in 
the prevalent tradition: Mechanism explains the behavior of animals, and 
the mind explains the behavior of man. According to Jensen (1962), Watson 
adhered to this dualistic position in his early years, and his conversion to 
a strict mechanistic position was due to the influence of Jennings, who was 
at Johns Hopkins University when Watson went there. Jennings’ classic 
work on the behavior of lower organisms (Jennings, 1906) had emphasized 
the physiological basis of the behavior of lower animals, but it also em- 
earning as an adaptive mechanism. 

Watson may have originally been a 


dualist, much of his early research had a strong mechanistic flavor. In 1903 
he published a large study correlating the learning ability of rats with the 
myelinization of their nervous systems. In any case, Watson very quickly 


established for himself the reputation of being a thoroughgoing mechanist. 
atson, 1914) he said that the study of the mind 


it is the realm of speculation and endless 
ychology. A science of psychology 
d the ultimate explanation of 


phasized the universality of l 
On the other hand, even though 


In a widely used textbook (Wi 
is the province of philosophy; 
word games. The mind has no place ans 
must be based on objective phenomena, an 
behavior must be found in the nervous system. 


Historical Background 


in Watson’s mechanistic philosophy, 
nor was there anything very new in the idea that psychologists should 
concern themselves primarily with behavior. This proposition was clear, 
though implicit, in much of the animal research published early in the 
century. The basic behaviorist doctrine had even been stated explicitly by 
a few writers (for example, Meyer, 1911). There was also a conspicuous 
failure of the older traditionalists to an 
Sigmund Freud, too, had begun to convince us that we often do not know 
why we do the things we do. Further failures of the traditional method of 
introspective analysis were reported by psychologists at the University of 
Würzburg. In short, the traditional distinction between animals as machines 
and men as rational beings was about to topple, and Watson was there at 


the right time to help push it over. 
Other very basic problems were confronting the psychological theorist in 
the early years of the twentieth century. There was the Darwinian concep- 


There was, of course, nothing new 


alyze consciousness effectively. 
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tion of continuity between animals and man. This idea of continuity m 
been Supported by Thorndike’s discovery of intelligent but ee 
behavior in animals, for example. The reconciliation of these discoveries 
with the traditional view of man, as well as with the traditional dichotomy 
between instinct and intelligence, produced an extensive critical tener 
There were a number of books with titles such as Instines and Intelligence 
(Morgan, 1912) and The Evolution of Animal Intelligence (Holmes, 191 1). 
It was also widely believed that modern techniques would soon St 
how the brain works, It was therefore necessary for the a mer ar 
theorist to formulate principles that would aid in the search for acaba 
of the brain, We have already noted that both Thorndike and Pavlov port 
quick to infer what goes on in the brain during learning. At a DE ie 
it was thought necessary to make psychological principles consistent u 
the ways in which the brain was Supposed to function. It was this zar a 
contradictory and mutually inconsistent assumptions that Watson's ea 
havioristic theory of learning was designed to untangle, This me 
reorganization was accomplished, first, by throwing the mind right out 2 
psychology and, second, by emphasizing the old concept of habit. Habit ses 
no longer considered simply a special variety of learning but was to se a 
as a model for al] learning. For Watson there was no problem about whate 
instinct left off and intelligence began, nor was there a problem about where 
awareness began to emerge in the evolution of the animal kingdom, 1 lie 
philosopher might ponder such unanswerable questions, but psychology 
was destined to be a science, which could be accomplished only, Watson 
said, by observing behavior and by relating behavior 

Watson's solution to the problems confrontin: 
sented in a series of lectures given at Columbia 
presented briefly in a paper in Psychological Review in 1913 
cussed at greater length in a book published in 
An Introduction to Comparative Psychology, achieved thr 
it showed how plausible it was to discard all 
phernalia from Psychology. Some of this cle: 
by means of translation; for e 
stimuli arising within the boc 


to the nervous system. 
8 psychology was first prer 
University in 1912. It be 
and then dis- 
1914. This book, Behavior: 
ce things. First, 
the familiar mental para- 
aring out was accomplished just 
xample, feelings and emotio 


ns became merely 
ly. Some of it was a 


Ccomplished by the writing 
of Physiological psychology. 


of awareness were attributed 
uscles in the body, When 
reports a thought, supposedly the stimulus to w 


is the feedback from muscles in the throat. T] 
or implicit speech responses and i 
responses as mental contents. The second achiey 
parative Psychology was a survey of the varie 
developed for the study of learning in animals, Anything 


hich he js really 


back from these 
f Watson's Com- 
ds that had been 
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know about the animal's sensory equipment, its motor capabilities, or its 
ability to associate sensory and motor events might be investigated scien- 
tifically by means of various kinds of apparatus and learning procedures. 


The third part of this important book was the presentation of the learning 


theory itself. Let us look at it. 


The Theory of Learning 


He proposed that when a stimulus 
he connection between them is 
rimarily upon 


Watson's theory was simple enough. 
at the same time t 
ng the connection becomes depends p 
and response occur together. 

in one respect: its claim that 


and response occur 
strengthened. How stror 
how frequently the stimulus 

This theory was like Thorndike’s theory 
learning consists of strengthening S-R connections. But there was a critical 
difference in the presumed learning mechanism; for Watson there was no 
law of effect, there was no reinforcement (which Thorndike had empha- 
sized), and there was a very important law of frequency, or exercise (which 
Thorndike had minimized). Watson's psychological principles of learning 
are readily translatable into his conception of the nervous system, The 
stimulus produces activity in a certain part of the brain, and the response 
is produced by activity in another part of the brain; when these two areas 
are simultaneously activated, by whatever means, the neural pathways be- 
tween them become strengthened. Two points should be noted in Watson's 
neurological model, One is that there was nothing new in the concept that 
repeated activation of a neural circuit strengthens it in the sense of making 
it easier to activate in the future. We have already found this hypothesis 
Stated by both Thorndike and Pavlov, and the basic idea had been formu- 
lated by psychologists and philosophers at least as far back as Herbert 


Spencer (1855). Probably everyone believed that was how the brain worked. 
i as a little more creative. He em- 


The second aspect of Watson's theory wa 

Phasized that learning does not produce (or result from) new connections 
in the brain, All neural connections hav 
of the animal’s original genetic constitutior 
threshold of an old connection or make funct 
It is interesting to note that today, sixty years lat 
how the brain actually changes structurally as a 


whether it changes at all. 
Watson's theory was primarily a protest against T horndike’s theory, and 


he was not above criticizing Thorndike at every opportunity. He said that 
(Thorndike’s law of effect was mentalistic. nothing but an updated version 
of the old hedonistic principle. In an attempt to refute Thorndike experi- 
mentally, (Watson (1917) trained rats to dig through sand to get into a goal 


e to be already laid down as part 
a. Learning can only lower the 
jonal one that has been latent. 
er, we are little wiser about 
consequence of learning or 
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box. Once in the goal box, 


z ; E E TO OE 
they were required to wait for the delivery 
food. According to Thorndi 


ke's theory (said Watson), because sitting A 
waiting are the last responses to occur, they, rather than the earlier a ome 
of digging which got the animal into the goal box, should be rein we 
by the delivery of food. But the digging response was in fact learned, isi of 
was taken by Watson as disproving the law of effect. We now gan 
course, that there are a variety of other possible mechanisms that can operi 


here. For example, digging in the sand may be intrinsically reinforcing, O! 
the goal box may become a secondary reinforcer, 

for the contemporary reinforcement theorist, 
if{the strength of learning de 
upon the simple freque 


There are many ways ou 
but at the time it are 
pended not upon the effect of the response DU 
ncy with which it occurred. lowing 
Watson’s law of frequency can be illustrated by means of the fol a 
example. Let us suppose we have a very simple situation in which an ae 
can make just two possible responses, one (R+) correct and the other (I a 
incorrect, Let us suppose for the sake of simplicity that R+ and = of 
of approximately equal strength initially. What happens over a set woke 
learning trials is that R— gradually drops out and R+ becomes the oe 
more likely response. This happens, Watson tells us, because, whereas aes 
may occur on half the trials, R+ must occur on every trial. R+ begon iĝ 
stronger than R— because it occurs twice as often. Then, as R+ begins ab 
acquire the advantage of frequency, it acquires also the added advantage 
greater associative strength to raise its frequency even higher.) 

Watsori also discussed a secondary law of recency: The most recent 
response is strengthened more by its frequent occurrence than is an earlier 
response) But the law of frequency was Watson's chief explanatory weapon 
He argued that it would operate to produce learning most effectively in @ 


r T 2 ý aa 
situation in which the correct response alters the Situation, It would operate 
i i » if R+ occurs, R- 


5 A «hich 
situation, in whicl 


The relation between Watson's theory and classica] conditioning is some- 
what more complex. In his 1914 book (Watson described many techniques 
for studying animal learning, treated as just 
another technique of investig: as requiring a 
der the law of 
3.1. Before the 


and Pavlov’s pro 
ation. It wa 


€ction is between 
l behavior, Again 


Further Developments 49 


orienting 
pe response 


CS tone a 


i 


Fig. 3.1. Watson's (1914) model of Pavlovian condi- 
tioning. There are assumed to be latent connections 
between virtually all stimuli and responses. Of this 
multitude only that connection is strengthened 
enough to become functional which feeds into a still 
stronger reflex, namely, the unconditioned reflex. 


etween the CS and R,. But at the same time that 
licit a variety of other behaviors and 
diffuse activities, including the UR. Because the particular association 
neurons that would bring about this diffuse discharge have been little used, 
no overt behavior occurs, and the CS produces only the R,. During the 
course of conditioning, however, the CS occurs and the UR necessarily 
occurs because it is elicited by the US. Consequently, the pathway between 
the two reflexes indicated in the diagram by the association neuron is 
strengthened sufficiently so that the CS will produce the UR. Basically then, 
according to Watson, what happens in a Pavlovian experiment is that the 
CS occurs at approximately the same time that the UR is made to occur, so 


the latent connection between the CS and the UR is strengthened. 
Implicit in Watson's analysis are two additional and rather strange as- 


sumptions about how the nervous system works. One is that the exercise of 
the US-UR connection can reduce the synaptic threshold of another neuron 
feeding into that reflex. The second assumption, which is implicit in his 
treatment of conditioning and especially in his treatment of the learning of 
motor sequences, is that there must be latent connections between all 
possible stimuli and all possible responses. If no new pathways can be 
created, then all potential pathways must already be present in the brain. 
Although neither of these assumptions appears really outrageous, both 
should be recognized as assumptions required by a Watsonian type of theory. 


we have a simple reflex b 
the CS elicits R, it also tends to € 


Further Developments 


Two events in the history of psychology contrived to change Watson's 


relationship to Pavlov. No sooner was Watson's 1914 book published than 
he was elected president of the American Psychological Association, and 
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he was therefore obliged to come forth with some shattering new develop- 
ment for his presidential address. At the same time, Bechterev’s famous 
Objective Psychology was translated into French (Bechterev, 1913), Let us 
recall that at that time Pavlov’s work was not available in English. But 
Bechterey had also developed, more or less independently of Pavlov, a 
variety of conditioning procedures such as shock to finger to elicit a motor 
response, finger withdrawal, rather than the salivary response Pavloy 
was working with. Although Bechterey presented very little data, his 
methods were described, and he presented a general philosophy that wes 
just as mechanistic and reductionist as Watson's own view of behavior. SO 
inspired, Watson set about to demonstrate classical conditioning in his own 
laboratory. He was, however, unable to accomplish salivary conditioning, 
according to the report of Lashley (1916). The finger-withdrawal condition- 
ing procedure of Bechterey had also raised a host of problems (Hamel, 
1919); nevertheless, a number of experiments did work, particularly when 
Bechterev’s procedures were appropriately modified. Watson was therefore 


: had ‘ i 1 
able to tell the APA that classical conditioning was a reality, the optimul 


P a Puls x De r 
Way to study learning, and that classical conditioning was the prototype fo 
all learning. American psycholog 


gy had begun to close the conditioning gap- 
What Watson had found in his successful conditioning studies was that 
a tone paired with shock came, after a few pairings, to produce both @ 
defensive reaction such as leg withdrawal and emotional responses such as 
altered heart rate. Watson did not distinguish between these two different 
kinds of behavior, apparently because of a conviction that all behavior 
changes are brought about by conditioning, Henceforth Watson identified 
himself as a conditioning theorist and Popularized the language and ideology 
of conditioning, so that when Pavlov’s work ultimately became available in 
1927 it must have already seemed vaguely familiar. American psychology 
thus entered a period of a decade or more in which everyone spoke of 
conditioning and paid homage to Pavlov, even though there was little or 
no understanding of Pavlov’s own work. “Conditioning” had come to mean 
simply the acquisition of S-R habits as a result of the stimulus and the 
response occurring together a number of times. 
Curiously, in his later works Watson backed away from the law of fre- 
quency as the universal mechanism of learning, His 1919 book, which was 
undoubtedly the most widely read and influential of 
the necessity for all behavior to be analyzed in S-R 
reality of the nervous system and the unre 
importance of learning, but offered no unify 
Indeed, (Watson said that the law of freque 
he afforded it no more discussion than the law of effect! 
But though Watson did not long remain an 
theory, he was constant in defense of his ge 


all his writings, stressed 
terms, emphasized the 
of the mind, described the 
ing interpretation of learning. 
ney is purely speculative, and 


ality 


advocate of his own learning 


neral view of behavior. For 
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example, he steadfastly rejected the mind, granting no place in psychology 
to any mental activity. (All mentai events were reduced to simple physio- 
logical processes) like internal stimuli, It is interesting to speculate on 
whether Watson’s unusual treatment of subjective experience was a result 
of his own peculiar subjective world. Herbert Langfeld reported (in a per- 
sonal communication) a conversation he had once had with Watson. Lang- 
feld asked him how he could possibly deny the subjective reality of images. 
How could he, for example, dismiss the reality of visual images in a dream. 
Langfeld said that Watson had settled the issue in one short sentence. He 
asked, “What is a dream?” It was Langfeld’s belief that Watson did not 
have visual imagery and that he therefore really did not know what some 
of his critics were talking about. Be that as it may, it is certainly clear that 
Watson emphasized kinesthetic stimuli and the theoretical part that they 
play in the determination of behavior. 

In an early experiment (Watson, 1907) he had run rats that were either 
normal, blind, deaf, or asnomic (with no sense of smell) in a maze. He 
found that in each instance the animal showed essentially the same pro- 
ficiency in learning the maze. He concluded therefore that none of these 
senses is an important part of the stimulus pattern involved in the maze 
habit. The critical stimuli therefore had to be from the one sense modality 
that he had not eliminated and that, for practical reasons, he could not 
eliminate: kinesthesis, the sense of the muscles. As Watson continued to 
develop the implications of behaviorism, unobserved kinesthetic stimuli 
were to play an ever-increasing part in his explanation of behavior, They 
e have seen, for the illusion of thought, for images, 


were responsible, as w 3 
all of what we think of as mediated behavior in 


and indeed for virtually 


humans. 
It is interesting to observe that, although Watson persistently demanded 


scientific objectivity in psychology, he had to rely incr a singly on kinesthetic 
feedback, the existence and properties of which were highly speculative. But 
on the other hand, it should be noted that stimuli feedback are not un- 
observable in principle, as are the contents of the mind. They are un- 
observable only in the practical sense that we do not have, or at least in 
his time Watson did not have, the instruments for observing them. Although 
we are still not very advanced in the technology necessary for measuring 
kinesthetic stimuli, there have been in the intervening years a number of 
demonstrations of the possibility that small implicit responses may be cor- 
related with subjective experience. Although we do not know if the sub- 
jective experience is caused by the muscle feedback, we do know that the 
requisite muscle activity is often present. For example, Jacobson (1932) 
measured muscle twitches in the arm and observed that they occurred in the 
appropriate muscles when subjects were instructed to “think about” throw- 
Ing a ball. More recently McGuigan (1973) found that when a psychotic 
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patient thought he was hearing voices, simultaneous muscle-action potentials 
were measurable in his throat. McGuigan suggests that the hallucinating 
psychotic may not be so crazy after all if he is, in some sense, actually hear- 
ing a voice. He is simply misinterpreting its source. The real test, of course, 
would be to stimulate the subjects arm muscles or throat muscles elec- 
trically in the appropriate fashion and to have him report thinking of 
throwing a ball or hearing voices. We are awaiting such evidence. er 
(Watson discussed instincts at some length because they are the origina 
behavior out of which learned behavior is built, Instincts are simply i 
connections, in some instances complex S-R connections, which have appre 
ciable strength prior to their first use. The structure of all behavior was, 
for Watson, reflected purely and simply in the structure of the nervous 
system and the S-R connections that it contains. The human being is 4 
tabula rasa, a blank slate, on which experience must write everything. The 
human is distinguished only by the fact that he has more latent S-R con- 
nections, so that he can display more different kinds of behavior and be 
responsive to more shades of stimulation. 4 
The idea that man is essentially nothing but the sum of his experience 
is an old one and has been defended by a number of important philoso- 
phers like John Locke and the associationists. 
humanitarian and liberal point of view because 
blame a man for any defect we may find in him. We must find fault 
instead with the world that has produced his offensive behavior. This 
message has become familiar to many through a recent book by Skinner 
(1971). The unique aspect of this message as Watson gave it is that it is not 
only psychologically necessary, as others before and since have claimed, but 
also physiologically necessary. It is demanded by the nature of the nervous 


system that ultimately must be the source of explanation for all our 
behavior. 


Probably/Watson’s most f 


It is in one sense a =“ 
it implies that we canno 


amous experiment was)the one reported with 
Rayner in 1920 in which(fear was classically conditioned in a child, the 


famous Little Albert. Little Albert was a healthy, normal infant who at- 
tended a day-care center. The CS was a white rat; Little Albert’s initial 
reaction to it was curiosity. He looked at it and reached out as if to touch it. 
The US was the sound made by hitting a heavy iron rai] with a hammer.’ 
It evidently made an awful noise because the UR consisted of Startle, falling 
over, crying, and crawling away. After three pairings of the rat and the 
frightening noise, the rat alone came to elicit various fear and defensive 
behaviors. After six conditioning trials the sight of the rat evoked a strong 


1 It appears from Watson and Rayner’s report that the ra 
was reaching for the rat. Although the procedure was osti 
pears to have been a response contingency; if so. 


il was struck only when Albert 
e ensibly conditioning, there ap- 
» it was actually a punishment procedure. 
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emotional reaction. Watson and Rayner then put Albert through a series of 
generalization tests. He was presented with a white rabbit, a dog, and a 
fur coat. In each case Albert showed considerable generalization of the emo- 
tional reaction that had been conditioned to the rat. When tested a few days 
later in a different situation, Albert showed little emotion, but one addi- 
tional conditioning trial brought out the fear in the new situation. Watson 
and Rayner discussed a number of procedures that might be used to de- 
condition or neutralize little Albert's fear, Although various kinds of therapy 
are discussed, Little Albert was unfortunately removed from the day-care 
center before any therapy could be carried out. 

It was concluded that the experiments with Little Albert show conclu- 
sively that directly conditioned emotional reactions persist, though with 
perhaps some loss of intensity, for a considerable period of time. Condi- 
tioned reactions were said to persist and to modify personality throughout 
life. Finally, it was noted that Albert was a stolid and phlegmatic individual, 
and it was suggested that had he been emotionally unstable to begin with 
he would have demonstrated much more emotional behavior. All these 
assertions were made, of course, in the absence of any evidence, but they 
were consistent with Watson's view of personality development. 

Characteristically, Watson could not resist poking a little fun at psycho- 

years later Albert might go to an 


analytic theory. He suggested that some à 
analyst to seek help for his strange fears and that he might even come to 


believe that his problem was the result of an unresolved Oedipal situation. 
But, said Watson, taking us into his confidence, we would know that his 
trouble was entirely caused by conditioning. So Watson had slain all the 
dragons and made psychology @ science. ; 
Watson's career in psychology ended tragically and abruptly in 1920. 
Scandal over a personal indiscretion necessitated his leaving his position at 
Johns Hopkins and soon leaving psychology altogether. He later wrote a 
book (Watson, 1925) and revised it, but this publication pt esented little that 


In the meanume, he had acquired an im- 


was new for learning theory- 
and some of the slogans and 


portant place in the advertising industry, 


advertising campaigns that can still be recalled were his efforts. 


Watson's Contributions and Impact 
atson had an enormous impact upon psychology. 


ould undoubtedly have proceeded 
it probably would not have 


In his short career (W 
Although the behavioristic revolution W 


quite nicely without his help and guidance, 
occurred so quickly; it certainly would not have been so spectacular. It was 


Watson, more than Pavlov or any other one man, who convinced psycholo- 
gists that the real explanation of behavior lay 1n the nervous system and 
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that as soon as we understood the brain just a little better most ae 
mysteries would disappear. He also made us believe that learning was sa 
all-important determinant of everything we are. Our behavior, our P 

sonalities, our emotional dispositions are 


all learned behaviors. 
Watson’s theory of lear 


s : = POretic posi- 
ning gave us a third important theoretical } à 
P s : x selgi as- 
tion. (His theory shared with those of both Thorndike and Pavlov oem 
sumption that learning is to be analyzed in terms of S-R connections. 


5 ses F . re systematic 
his position was vastly simpler than Pavlov’s and much more syste 


than Thorndike’s.) But then, paradoxically, it was largely because of sso 
and his influence that much of the language and some of the ee as 
procedures originally Proposed by Pavlov became a standard part of ani 
learning literature. And it was mainly because of Watson that so many 


$ P . oak vas. sO 
Psychologists came to believe that what they called conditioning w 
important. 


References for Further Reading 


5 his 
Watson’s own major works make Provocative and enjoyable reading. te 
books, Behavior (1914) deals with animal and comparative psychology, Psych rim 
from the Standpoint of a Behaviorist (1919) goes on to humans, and Behavio 


A has 
Tt was Weiss (1925), who ! 
Fenn est an 
mechanism's staunchest a 


CHAPTER 
4 


GUTHRIE 


o 


(Edwin R. Guthrie (1886-1959) was an important contributor to the 
development of learning theory, even though he did very little research 
himself and his theory instigated very little research {Initially his theoretical 
position was much like that of Watson, but it evolved over the years and 
developed a unique character of its own.)Guthrie was primarily a teacher, 
and for a period of forty-five years he devoted much of his effort to the 
indoctrination of undergraduates at the University of Washington in the 
ways of psychology. 

Guthrie originally prepared to bea philosophe 
enough in the experimental aspects of psychology to see that many of the 
problems that had traditionally been regarded as philosophical could be 


r, but he became interested 
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better approached experimentally, 
1959) that(he was much influence 
Singer? Singer had claimed that m 


He says of his own training | me 
d by the teachings of the philosop ië 
any of the traditional problems gee 
slated immediately into problems of behavior and pee yi 
at that level. But the behaviorist revolution was in full swing, and gee 
Guthrie’s first works appeared they were strongly mechanistic, = ai 
Guthrie (1921) wrote an introductory psychology textbook that Ae oa 
very largely on the foundation that had been laid by Watson) se a 
behavior Was explained in terms of simple S-R connections, and ie AeA 
behavior was explained in terms of a multitude of simple S-R Se . 
There was little innate organization in the theory, and there were hea). If 
tivational principles, he flavor of the work was strictly Watsonia. ne 
there was any difference, it was that Smith and Guthrie had somewhat mo 
appreciation than Watson did of the complexity of behavior. . hey 
Smith and Guthrie recognized two kinds of learning. One, which : A. 
called “positive adaptation,” was simply a recasting of Watson's wie ven 
frequency: The more frequently a particular response occurs in a oly 
situation, the more likely it is to be repeated in that situation) Rdlanv d 
little was said about the underlying mechanism; recognition of this ae 
of learning was little more than acceptance of the traditional concept © 
habit, which traditionally reflected only the frequen 


(The second kind of learning was a little more 


Guthrie called it conditioning, but it bore little r 
that Pavlov studied. As Smith and G 


initially elicited by specific 


cy of occurrence. i 
interesting, Smith ane 
semblance to the learning 
uthrie described it, all responses are 
unconditioned stimuli, but a response 
become “conditioned” to a previously ineffective stimulus The essence 
conditioning is thus the substitution of the CS for the US in eliciting @ 


response, In one sense, all learning has to fit this formul; 
formerly did not occur in a giv i 


or some part of it can be r 
not call learning 


now does, then the situation 
a substitute stimulus.) Today we do 
unless jt involves particular kinds of 
responses and unless it occurs under Particular Conditions, for example, as 
a specific result of pairing CS a such subtleties were not recog- 
nized in the 1920s, and for Some writers, including Smith and Guthrie, 
Conditioning was virtually synonymous with learning, The word did not 
mean a particular kind of learning; it was merely 4 | 
writer's faith in the objective method, the reality of t 
and the explanatory power of the S-R association, Aj] 
explained simply in terms of its eliciting stimuli: 
principles were to be allowed. Specifically, learning i 
than a change in the eliciting stimulus. 

Whereas many of our 


egarded as 
“conditioning” 


abel signifying the 
he nervous system, 
behavior was to be 
no other explanatory 
Nvolved nothing more 


learning theorists developed their ideas in protest 
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g ideas, Guthrie was more conservative. His first_theory, 
vith Smith, was little more than a restatement of 
as, let me add, an attractive and influ- 


against prevailin 
written in collaboration ¥ 
Watson's behavioristic position) It w: 
ential statement, but its main effect was to extend and prolong Watson's 
views of behavior in general and of learning in particular. This simple, 
S-R, and mechanistic view of behavior was also to provide the background 
for Guthrie's own highly individual theory of learning, which first appeared 
a curious situation: The conceptual back- 


in 1935. Guthrie was thus in 
uis own earlier 


ground for his 1935 theory had been provided in part by l 
efforts, which had kept the mechanistic tradition going. As we look at the 
1935 theory, we should see it as a direct descendant of the mechanistic 


tradition begun by Pavlov and carried on by Watson. 


The Theory of Learning 


features of Guthrie’s 1935 theory is that, 
whereas all other theorists before and since have emphasized the gradualness 
of learning “and conditioning, (Guthrie maintained that_learning occurs all 
at once. A given response becomes connected with a given stimulus in one 
trial. This assumption clearly contradicts the obvious fact of gradual ac- 
quisition observed in most learning tasks) but Guthrie evaded this difficulty 
by redefining the stimulus. We may view the whole situation as a stimulus, 
but in reality the experimental situation consists of a mass of stimulus 
elements that change from moment to moment. Millions of nerves pour 
sensory information into the brain at every instant. Visual stimuli change 
as the animal moves about the apparatus. Olfactory and auditory stimuli 
that may be irrelevant to solving the problem are nonetheless present in 
the total pattern of stimuli. There are additional stimuli arising as kines- 
thetic feedback from the animal's own muscle contractions as it moves 
about. (The total pattern of stimuli is thus constantly changing ( According 
to Guthrie, when a response occurs, it immediately becomes conditioned 
to whatever stimuli are present at the moment. The formation of this con- 
nection constitutes the learning that occurs on that trial. On the next trial 
a quite different stimulus pattern may exist, and the response may not be 
elicited. But if the response occurs again for some reason, it will be condi- 
tioned to the stimuli present in the new pattern. Over a series of trials the 
response gradually becomes conditioned to an ever-increasing population of 
possible stimuli, and as this happens the response becomes ever more prob- 
able. After a number of trials, when the response becomes connected to 
virtually all the stimuli that can occur in the situation, then the response 
will be almost inevitable. The gradual improvement in performance char- 


One of the most distinguishing 
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acteristic of most learning experiments is thus seen to a 
plexity of the stimulus world. 


certain; it is the world that is c 


ise from the ei 

> Bac ia ae le an 
The learning process itself is simp 
omplicated and uncertain. Tiii 
imulus clements that appear On _ er 
us elements that persist over trials, L 


» for example, that we h 


color of the apparatus is constant and irrelevant. Consider then the ol 
tionship between this constant stimulus element and the ongoing Sien dt 
behavior, the different responses the animal makes in the situation. / mai 
walks, looks, orients, grooms, or explores, these responses berang poe 
tioned to the total stimulus pattern that exists when they oc am W me a 
means in the case of an irrelevant stimulus such as the color of the Tiie 
ratus is that if one of these responses is at one instant conditioned to tai 
color, then at the next instant, when another response occurs, the p! i 
learning will be undone and the new response will become conditioned to it- 
(Thus we have the Strange concept of learnin 
€ver the animal is engaged in 
tached to prevailing stimuli, 
and replaced by the le 
learning not 
tinuing strea 


8 occurring all the oe 
any behavior, the response is one eae ae 
But much of this learning is promptly nie veil 
arning of the next response. (In short, Guthrie ev “i 
as a special event that occurs once in a while but as a ome 
m of minute connections being formed all the time. Lise nieta 
we make any response, we are conditioning it to prevailing stimuli and u 
conditioning some prior response. } 

There is another impor 
Guthrie gave no actual figu 
a million) sti 


uthrie’s analysi Although 
ged to be a great many (perhaps 
898 Upon the animal at any instant) This 
ation into microscopic stimulus elements means that the 

elements to which the response becomes attached) 


. è a zraig fais 
the physical things we cal] stimulus objects. I 
o determine from Guthrie’ 


ecified. So, in spite of the 
ere isa serious question 


] T 
s theory how the the! f 
apparent objectivity O 

REN ê 
about how objective th 


a remarkab 
d to expl 


re impos 


le correspondence 
ain learning and the 


ed upon animals. In 
To illustrate the point, consider 


experimental procedure could be 
the stimulus, the response, and of the reponse are ahi well 
defined as observable events. In Thorndike’s theory the S* strengthens the 
connection between the S and the R. Thus the increased probability of the 
observed response in the experir Nn was assumed te ie Neen 


Paradigm in which 


mental] situatio: 
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by an underlying neural connection between the stimulus and the response. 
Pavlov's experimental paradigm was different from Thorndike’s, but Pavlov 
included the same kind of correspondence between experiment and theory. 
He paired stimuli until the response to one began to occur to the other; 
then the new reaction was explained theoretically on the assumption that, 
through pairing, the stimuli had become equivalent. The theoretical pair- 


ing mirrors the procedural pairing. In Watson's theory learning consisted 
hat happens in a Wat- 


of new S-R connections, and, again, if we watch w 
sonian experiment we see that a stimulus and then a response occur. The 
empirical fact that the response becomes increasingly probable in the 
presence of that stimulus was attributed to the strengthening of an S-R 
connection, Again, the theory reflects the data of the experiment. All these 
theorists had proposed theories that, in effect, simply redescribed their data. 
But Guthrie was different. He was describing not immediately observable 
events but a world of his own imagination. (There are millions of stimulus 
elements, a few thousand of which become conditioned on any one trial, 
Earlier theorists may have been overoptimistic about the simplicity of the 
learning process and of the various processes, such as learning, that they 
thought controlled behavior. But by the 1930s it had become apparent that 
behavior is really rather complex and is governed by many different kinds of 
determinants. It is as if behavio intrinsically unpredictable, so that 


r were 1 
there has to be some part of any viable theory that is complex or uncertain 
enough to prevent us from making 


direct predictions of behavior with it! 
We may be able to predict the average case; we may 


be able to predict the 
rate of behavior that occurs over 


a period of time, but it seems unlikely 
that we can ever develop models that will tell us precisely what a particular 
Organism will do at a particul 


ar moment. The remedy appears to be a 
theory that is intrinsically probabilistic or 


a theory that introduces unpre- 
dictability into the causal analysis of behavior. 

Unpredictability may reside in the syntax of a theory (how the terms of 
the theory are hypothesized to relate to each other) or 1n the semantics of a 
theory (how the theoretical terms are related to observational data). Now 
(the syntax of Guthrie's theory is simple and straightforward: Responses 
become conditioned to stimulus elements. That is all there is to it. The 
complexity and the inherent unpredict > 


ability appear in the semantics. 
Recall how important kinesthetic feedback stimuli were assumed to be to 
Watson's theory. They were necessary 


in his theory (for example, to explain 
awareness), but they were not observable. G 


uthrie seems to have taken this 
principle of unknowability one step farther. If no one can know what 
stimulus elements are going to impinge upon an 


animal at any instant, 
Guthrie can assert that, if the elements were known, 


then we would find 
them simply and directly related to the animal's observable behavior. In 
practice we do not know what elements are pr 


esent, so behavior remains 


Mv 
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unpredictable, I 
theoretical synt 
and perhaps u 
plexity and un 


simple 
5 ie retained the simp 
n summary, we can say that; Guthrie panne sult 
~ $ + a x 

ax of Watson, but he made thé semantics comple 


F m- 
: i i lous COl 
ntestable enough, to be consistent with the oby 
predictability of behavior, ) 


Applications of the Theory 


FA (One Phenomenon th 
generalization | For Gut 


as “similar” to the tr 
dimension, 


at follows neatly 


ior is 
= See n f behavior !5 

from Guthrie's view of beha 
hrie it w 


z s 
as not necessary to think of the test eke 
aining stimulus or as related to it on some ogre 
Because all stimuli are constituted of a number of di 1 ele 
elements, seneralization can be attributed to the number of apnea! Jle, 
ments shared by the training situation and the test situation) For peed 
if an animal is traine he presence of, let us say, a mi 
discrete bits of se i 


n 
i aed ; patte! 
sensory information and then encounters a stimulus I 

that contains only a half-million of the 


responding has to be somewhat reduced, 
a quarter-million of the ori 
response must be further 
cepts like simil 
Unfortunately, 
quantify the nu 
quantification i 
had the 


Same elements, the probability on 
In a test situation containing tr 
s elements, the probability of ane 
» according to Guthrie, v: Bue nn 
Precise notion of partial identit} 
argely illusory b 
al elements, Our 
Of stimulus simil 


ginal stimulu 
reduced. Thus 
arity are replaced by the 
this Precision is ] 
mber of identic 
S still in terms 
conceptual advantage that g 
phenomenon Separate from learning itself. ( XE 
parcel of the learning Experiment, simply because from one trial to the ee 
there could be only partial identity Of the total stimulus patterns. In effects 
any learning experiment really consists Of a series Of generalization trials? i 
The analysis of discrimination learning follows immedi D 

i becomes conditione 
elements, while another response is becoming 
ut there is an ; complication here: 
Cans of its own be- 
eyes and the ears, 
hat the animal learns. 
Timination may be the 
“PProaching the correct 
Tat has learned to look 


ay to 
yal 
ecause we have no W Ya 
janet st 
best approximation to 


: vate analysis 
arity, But Guthrie's analy 3 


eneraliz 


‘ iewed as 
aton was not viewed 


onal ave and 
reneralization was part 


another set. B 
an animal can produ 
havior, particularly orienting ri at direct the 
these orienting responses may be iti 

The proper orient 


Because 


hite disc 
n, and 


nce the 
at it. 


(There is no reinforcement mechanism in Guthrie’s Systenty the: ronse 
X a res e i assumed to hay 
quence of a response is not 


© any direct influence on the 
strength of the response. The apparent “SPonse-strengthening effect of 
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reinforcement is explained by Guthrie, as are many behavioral phenomena, 
in terms of change in the pattern of stimulation. Let us suppose that an 
animal is in a situation S, in which no response has been conditioned to 
enough stimulus elements to make it occur regularly. Under these conditions 
a series of different responses will occur, each one will become conditioned 
to the elements existing at the moment, and then each response will become 
unconditioned as it is replaced by the next response. (The different re- 
sponses occur, of course, because they are elicited by new stimulus elements 


that appear often as the results of the animal's behavior.) This succession 


of responses can continue indefinitely. But let us suppose we have arranged 


the experimental situation so that when one particular response, R,, occurs 
the situation is so changed that the animal is no longer in S, but in a new 
situation, S,. Thus if R, is programmed to produce a dramatic change in 
the situation, the connection formed between S, and R, will be preserved. 
annot be conditioned to the elements of S, because these 


Further responses c: 
no longer exist. 


elements, or at least a substantial proportion of them, 

A reinforcer is therefore merely an event that produces a substantial 
change in the total stimulus pattern, Encountering food in the goal box or 
in the hopper of the Skinner box and opening the door of the puzzle box 
are presumably dramatic enough changes in the stimulus situation so that 
the immediately prior response remains connected to the immediately prior 
stimulus situation. Maze learning can be explained in the same way; a cor- 
rect response moves the animal from one segment of the maze to the next 
where there are a number of different stimulus elements. So the correct 
response is learned. Let us suppose that, instead of making the correct 
response, the animal makes some competing response, such as grooming. 
Grooming would then be learned, but, when the stimuli that initiated the 
grooming are gone, it will stop and be replaced by some other response. 
Only the correct response can change the situation. Whereas other competing 
behaviors come and go, only the correct response will become gradually 
conditioned to more and more stimulus elements—because it is consistently 
the last response to occur in their presence. Because Guthrie’s theory has no 
need for a reinforcement mechanism, because learning is assumed to depend 
only upon the stimulus and response occurring together, it is called a 


contiguity” theory. 


(According to Guthrie's analysis. it should make little difference what 


kind of event produces the stimulus change: an electric shock, for example, 
should be just as effective a change as the presentation of food. But we 
know that, if electric shock is made contingent upon the animal's running 
into the goal box, it will quickly stop running. So there are instances in 
which stimulus change fails to protect the prior S-R connection and actually 
produces behavior that effectively competes with the response and prevents 
its further occurrence. Guthrie was well aware of this problem and dealt 
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ying 
with it in his treatment of punishment. But he also considered sine An 
mythical experiment (one that was discussed but apparently kaas er n lod 
animal runs down an alley, and as it comes to a certain point a 7 r one 
opens, and the animal drops out of the apparatus and into a tuun aning 
is wholly different from the original alley. Guthrie predicted that p allel 
in this situation would be learned, and he described a number o a of 
situations, common in every-day life, in which we acquire habitual e: 
responding even when the consequences of our behavior arc ee iis 
ae conception of learning as a process that is conmuranina lt) E j 
on and continually being undone leads very simply and directly let us 
ose that in a given EA awe 
a certain response, R,, to obtain food. We honing 
changes the situation (making ts ig 
and R, is preserved. In extinction as Jus 
no more food, so there is no More S,; therefore there is no more ani 
rR, connection, Accordingly R, becomes Tepe 


. sly lez ned 
€ grooming and exploring. So the previously lear 
hen food is withheld, * 


Seen that the present 


by other behavior lik 
response is lost w 


/ Motivation 


i P -oming 
“ah ings was becom 
an important aspect of behavior by the 1930sGn purely associative tert 


; i stimuli. 
ational concepts pgese, only a special new class of stimu 
Ogical condition such 


as hunger or fear there are, he mi 
characteristic internal stimuli. These stimuli are unique in that they âf 
; mal moving about in parmi 
l engaging in some specific cOn 
* arising from hunger can pe 
arn to eat when it is hungry 
renal stimuli? The animal also 
8 because the learning 
change that food makes in the 

that makes an animal appear 


age, scratching at the 


apparatus, and many at look like F 


of food—occur because they too have be 
hunger stimuli? 


i. With the postu- 
n whatever behavior 
T a concept of drive, 
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instinct, or any other kind of motivational principle. Again we can see that 
a whole class of behavior phenomena—iotivation—is explained in terms 
of the complexity of the total stimulus pattern. H 

We must consider one further application of Guthrie’s theory. Suppose 
we have two situations, S, and Sẹ in which the animal is required to learn 
R, and R, respectively. If the two situations are quite different, with few 
common stimulus elements, the animal will just acquire the two responses, 
each independent of the other. But let us suppose that there are many 
elements in common between S, and S+. Then each response will tend to 
occur not only in its proper situation but in the other situation as well. 
Suppose, for example, that an alley is similar to the goal box and that R, 
is eating in the goal box. If food were suddenly introduced in the alley 
we would expect the animal to have a tendency to eat it because of the 
tendency for the eating response to be elicited by alley cues. In the absence 
of food in the alley there is still some predisposition for the animal to 
make the eating response or whatever other response occurs in the goal box. 
We can call such behavior anticipatory responses. They can result in a 
short-circuiting of a sequence of responses. Let us suppose that there is 
a long series of behaviors that the animal must produce, as in running a 
maze; then there will be a tendency for later responses in the sequence 
to move forward, so that the rat will make anticipatory errors. A number 
of writers since Guthrie have developed similar views of anticipatory be- 
havior and the role that this type of short-circuiting mechanism can play 
in behavior. When we discuss Hull in Chapter 6 we will see that he and his 
followers have attached tremendous importance to such a mechanism. The 
principal use Guthrie made of it was in his interpretation of punishment. 


Punishment 


We have noted that earlier theorists had not come to any consensus on 
the mechanisms involved in punishment and that there was little under- 
standing of why the punishment procedure was sometimes extremely effec- 
tive and at other times quite ineffective in weakening behavior. Guthrie 
offered a plausible explanation of these discrepancies. His interpretation of 
Punishment can perhaps best be illustrated by one of his own examples. 
Guthrie considers a modern, liberal young mother who decides to raise her 
child permissively; she will not use punishment unless she really has to. 
But, she decides, one rule is inviolate and must be backed up by punish- 
ment: The family lives on a busy street, and the mother decides quite rea- 
sonably that the one punishable offense will be the child's walking out on 
the street. So the mother coolly and rationally decides that such behavior 
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will be punished in the child’s own best interest (children are always pun- 


ished in their own best interests). One day it happens. The mother sees her 
child running toward the street. She chases him and catches up with him 
just in time to slap him. Having established the rule, the mother has no 
question about whether to slap the child—that has already been settled; 
but there is one very important, crucial question, according to Guthrie. 
Where should the child be slapped? It is tempting to slap him on the rear 
end because that is convenient when the mother has run up behind him, 
but it would be exactly the wrong thing to do. Guthrie observes that what 
the young mother must do is to reach around in front of the child and 
slap him in the face, The reason is that a blow in the face will make him 
recoil, and the recoil response will then become attached to all the stimuli 
presented by the street. If the child is hit on the behind he is likely to lurch 
forward, and if that response is learned then sooner or later the child will 
be lost on the street. 

Again we can think of the chaining of responses. The S, situation is the 
sight of the street in the distance, and there is some initial inclination to 
make R,, approach, to this stimulus pattern. S, is the stimulus elements 
presented by the street up close together with the slap, and R, is the recoil 
reaction. R, is immediately attached to S, because of the great stimulus 
change. Now, to the extent that S, contains similar stimulus elements to Sr 
the recoil reaction will generalize back to S, and will effectively compete 
with the originally elicited approach behavior. Guthrie thus tells us that 
punishment will be effective in weakening some behavior if and only if 
two conditions are met: First, the punishing stimulus 
response, R,, that is incompatible with the punished res 
ing from a slap is incompatible with 
stimulus situation that elicits the com 
of stimulus elements with S,, 
occurs, 


itself must elicit a 
ponse, R,, as recoil- 
running forward; second, S., the 
peting response, must share a number 


the situation in which the punished response 


As with most of the predictions from Guth 
of his analysis of punishment have generated 
one study, reported almost thirty years later by Fowler and Miller (1963). 
provides rather nice confirmation of his analysis. Fowler and Miller trained 
groups of rats to run in an alley for food. A control group simply ran and 
obtained food. The learning curve shown in Figure 4.1 reveals an orderly 
reduction in running time on successive trials. One experimenta] group 
received a mild electric shock to the front feet just as the animals reached 
the food cup. These animals showed retarded running relative to the con- 
trols; they seemed reluctant to run. The traditional view of punishment 
suggests that, with some positive reinforcement and some Punishment, the 
rat should compromise and run more slowly. But the important group 


rie’s theories, the implications 
relatively little research. But 
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Fig. 4.1 
oups received either 


shocked near the goal. The gr 
hind-paw or front-paw shock; controls received no 


shock. (From Fowler & Miller, Facilitation and 
inhibition of runway performance by hind- and fore- 
paw shock of various intensities. Journal of Com- 
parative and Physiological Psy chology, 1963, 56, 801- 
805. Copyright 1963 by the American Psychological 
Association. Reprinted by permission.) 


s that defied the traditional view. These 


rats were shocked on the hind feet just as they reached the food cup. Their 
performance was facilitated relative to the controls. All conditions were the 
same for the three groups except whether they were shocked and where the 
shock was applied. It turns out that the rat's reaction to shock to the front 
feet is to recoil, to pull back. The slower running is predicted from Guthrie's 
theory, not because the rat is in conflict when there are both food and 
shock at the end of the alley, but merely because it is subject to response 
competition. It is partly running because of the food and partly pulling 
back because of the shock. On the other hand, the reaction to shock to the 


hind feet is to lurch forward. As the rat traverses the alley, it is partly 
lurching forward because of the shock. It 


sults reported by Fowler and Miller prob- 


Fowler and Miller ran gave result 


running for food and partly 
should be emphasized that the re: 
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ably depend upon using exactly the right kind and intensity of shock. Ce 
tainly a more intense punishing stimulus would inhibit all running. None- 


: 3 : . ERT Tabie 
theless, the results provide nice confirmation of Guthrie's remarkabl 
prediction. 


Guthrian Experiments 


Guthrie himself was not primarily an experimenter, but two AOE 
experiments have been conducted to test his theory, and produced kon 
support for it. Learning should occur more rapidly in a situation in which 
there is great constancy of the total stimulus pattern from one trial to ie 
next. A classical conditioning experiment in which all stimulus events are 
carefully controlled should therefore produce faster learning than a free 


x $ Š 3 š 3 wi Jus 
Operant situation in which there is more opportunity for random stimu 
variation to occur. Voeks ad 


ment and took special prec 
conditions. Even the subje 


954) conducted a classical conditioning CEREN 
autions to ensure the constancy of the stimulus 
ct’s internal stimulus environment was held cars 
stant; Voeks used a system of ready signals so that the subject could initiate 
each trial when he felt properly relaxed and ready for it, Using a bell as @ 
CS and an eye blink as the CR, Voeks found very rapid acquisition of the 
conditioned response in each subject. A typical subject might show pe 
conditioning at all for the first six trials and then on the seventh trial show 
the CR in full strength. Most subjects demonstrated at some point a sharp 
transition between no conditioning and full conditioning, 

Voeks’ results raise an interesting point. 
look at the performance of individual subje 
we found that different subjects showed 
numbers of trials, as shown in Figure 4.2, Suppose that now we were simply 
to pool the results from the different subjects to det 
formance for the group. We would obtain 
like that in Figure 4.2, which would look y 
of any one subject. The rapid transition 
gests that the typical gradual learning 
statistically averaging the performances 
Stages in learning. Certainly the average r 
acquisition in individual subjects. 

A second experiment which provided direct Support for Guthrie’s theory 
was reported by Guthrie and Horton (1946). It was a variation of the old 
cat-in-the-puzzle-box trick. There was a th a door that had to be 
opened, and food was just outside. Proj up from the center of the 
floor was a pole, which was ingeniously arranged so that when it was tilted 


rere tO 

Let us suppose that we were S 
cts in a learning experiment ant 
ia a rent 

a sharp transition after differen 


ermine the average per- 
a group-average learning curve 
ery little like the learning curve 
al learning found by Vocks sug- 
curve may often be an artifact of 
of subjects that are at different 
esults for a group need not reflect 


box wi 
ecting 
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Ss 


Performance 


Trials 


Fig. 4.2 Diagram showing how a number of sub- 
jects all showing one-trial learning, and some vari- 
ability, could produce a relatively smooth and con- 


tinuous group learning curve. 


ould happen: The door would open so that 


i acs a 
n any direction two things w 
vas photographed at the 


the cat could go out and eat, and the animal v 
instant the criterion response was made. The data from the experiment con- 
Sisted of a series of pictures of the response that occurred on each trial. 
These pictures showed a peculiar fixity or stereotypy to the animal’s be- 
havior, If we know how the cat moved the pole on trial 13, then we can 
make a very good guess how it will move the pole on trial 14: It will make 
precisely the same response. In one series of trials the cat might bump the 
pole with its shoulder. Suddenly the behavior changes, and there follows a 


series of trials in which the pole is hit with a paw. Then this pattern is 
replaced by bumping with the head. The interesting point is that when 
the pattern changes it changes all at once. There is no gradual transition, 


and there is no gradual acquisition of skillful movements. We do not know 
what produces the transitions. Why is a particular cat a shoulder bumper 
on one series of trials and a head bumper on the next series of trials? Pre- 
sumably the stimulus situation changes in some way. Although the shifts in 
response patterns were difficult for Guthrie to explain, the stereotyped 


nature of the behavior over a series of trials provided support eae eee 
view of learning. 

Guthrie was inclined to emphasize the fixity of behavior both in reporting 
this research and in many of his popular writings. He appeared to be 
fascinated by the fact that we often do stupid and maladaptive things. Our 
behavior often fails to reflect our intelligence: often it appears to be stereo- 
typed and mechanical. The cat in the puzzle box does not come to respond 
with appreciably greater speed or greater efficiency or more finesse over a 
series of trials. We find instead that it makes the same muscle contractions 
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On successive trials. This mech 
Guthrie's theory, just 
havior have given sup} 


é . DOE PE, to 
anical type of responding gave uea 

k eS SURRES Je- 
as the apparent flexibility and adaptiveness of 
port to other psychologists’ theories. 


Guthrie's Contribution and Impact 


5 š 5 a n -arry out 
Guthrie himself did little research, and he had few students to carry 
research on his ideas. 


Although he taught great numbers of students, a 
were mostly undergraduates (there was no Ph.D. program at the University 
of Washington during most of Guthrie's yea 
real disciples to carry forth his ideas, < 1950). 
spoken defenders of the theory (for example, Sheffield, 1949; Vocks, 19% 
but their efforts initiated little further activity. 


- : s n ¢ had relatively 
There are perhaps two main reasons why Guthrie’s work had relat 
little immediate impact, 


One was that it generated little research. ae 
second reason was that when the theory was formulated in 1935 it sige 
well outside the mainstream of developments in j 
1930s many theoris 


ts had begun to 
nomena of motivation: 


; ; » have any 
s there). Nor did he have a h 
ee > out- 

A few former students became © 


behavior theory, In : 
attribute great importance to the phe 
theorists such as Tolman and Hull could not ere 
ceive of principles of learning without supplementary principles of mot 
vation (see Chapters 5 and 6). But while this ne 
behavior theory, Guthrie was boldly proclaiming that motivational con- 
cepts are not necessary for the explanation of behavior. ate 
be simply a different class of stimuli, During the 1930s Thorndike’s law’ 0! 
effect was being reinstated as a powerful principle of learning: theorists 
such as Hull and Skinner were soon to make it the fundamental principle 
of learning, But while reinforcement was becoming theoretically important 
Guthrie was proclaiming that we did not need a concept of reinforcement. 
A reinforcer is simply € of stimulation. While other learning 

asingly concerned with semantic questions. 
mulus and response, ( 


matters. He w 
response unit; a response could be 


or a molar event like driving dow 
were complicating 


A aji into 
w Impetus was entering 


2 ai o 
Motives were said t 


a chang 
theorists were becoming incre 
for example, how to define sti zuthrie was relatively 
unconcerned about these as not committed to any specific 
a small thing like the twitch of a muscle 
ntown. Theorists like Tolman and Hull 


the syntax of their theories in order to deal with the 
complexity of behavior, but Guthrie ret 


ained the simplest syntax and put 
all the complexity into the definition of the stimulus. } 

Thus Guthrie's theory had an anachronistic quality when it first ap- 
Petey fo Sane eee ie quality was more striking when the second edition 
of his book appeared, with little change, in 1952. In ONA respec stool 
as a direct continuation of a theoretical program inidaiea by peta 
continued by Watson. Guthrie shared wi 


th these earlier theorists the con- 
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viction that all behavior is to be explained purely in terms of S-R associa- 
tions. He included no motivation, no reinforcement, nothing mentalistic- 
sounding, nothing but Ss and Rs, especially Ss. 

But in some respects the theory has had considerable impact. Guthrie's 
analysis of extinction, punishment, and other phenomena in terms of com- 
peting responses has been widely accepted. His theoretical formulation 
remains the definitive statement of a purely S-R interpretation of behavior 
and of a pure contiguity theory of learning. As such it has considerable 


intellectual interest, regardless of how effectively it can be applied to prac- 
n be applied to theoretical problems. 


contiguity position, Guthrie's theory 
theoretical development of con- 
ground for subsequent theo- 
ather than to 


tical problems or how adequately it ca 
And, as the definitive statement of the 
has provided the framework for further 
tiguity theory; that is, it has provided the back 
retical models in which learning is attributed to contiguity r 
reinforcement, Let us consider two examples. 


Later Developments 


William K. Estes was not a student of Guthrie's (he originally worked 
with Skinner), but in 1950 he proposed a mathematical model of learning 
an learning principles and that has had 


that embodies a number of Guthri 
the effect of greatly expanding and clarifying Guthrie's original position 
and generating a good deal of research. One of the basic assumptions in 
Estes’ model is that there is a large population of potentially effective 
stimuli that can impinge upon the organism but that at any one time only 
these stimuli is actually present. The subject, in 
from the population of stimuli. Estes 
a fixed proportion of the total (usually 
at these samples are drawn ran- 
a given element on one trial 
does not depend upon what happened on the previous trial. Another basic 
assumption in Estes’ model is that a response is immediately conditioned 
to all stimuli that are sampled when the response 


trial. 


a small proportion of 
effect, is constantly drawing samples 
assumed that this sample constitutes 
designated by the Greek letter 6) and th 
domly, so that the probability of sampling 


occurs, that is, on each 


Suppose then that on the nth trial there is some definite probability P, 
that the subject will sample one of the stimulus elements to which the 
response R has already been conditioned. This probability, P,,, also has to 
be the probability that R will occur on the nth trial. Certain specific im- 
plications about what will happen on the n+ Ist trial follow. There will be 
an increase in response probability, that is, Pugs — Pa will be positive, if 
the response occurs on the nth trial and a randomly sampled element is 
not already connected to R. The probability of R will then increase by the 
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if R occurs on the nth trial. If R fails to 
ill be a loss in the probability of the seit 
an element to which R is already conditioned. Th 
loss will be Peay = Pe = —6P,, if R does not occur on the nih trial. ee 

These two Equations constitute the body of Estes’ model. wh ae 
lying Psychological principles are basically Guthrian, but the chee of 
makes of the model is quite unlike a Guthrian analysis. The probabi 1 ia 
a response at a particular time is an objective datum, and the data ae 
experiment may or may not be consistent with the equations, so Fs 
model is empirically verifiable, ; adicted 
Stant; it sometimes turns out that P, does not increase in the prec ae 
manner; sometimes the model does not predict accurately WAE ee 
following a correct and an incorrect response. Furthermore, Estes on | 
can be manipulated to yield a variety of implications that go far cent 
the kind of Situation from which they were derived and that are there 


“tions 
á Pio ý 3 Iie AAO ~ductiol 
not intuitively obvious, Thus, Estes has teased out a variety of de 


; of 
age a ete y à s ~~ vo classes O 
about discrimination learning in which two responses to two cl 
stimuli must be learnec 


l, and about probabilistic learning when ak 
Sponses are reinforced with different probabilities. In the latter case igh 
theory leads to the prediction that the response probabilities aon MoET 
the reinforcement probabilities. Such matching is often (but not a Ke 
found, The results thus sometimes confirm the basic assumptions anc 
mathematical derivations. aig 
Although Estes’ basic psychological assumptions about the lear spi 
process are very Guthrian, his method of developing these assumptions 81 a 
him two enormous advantages over Guthric’s original method of ennaa: 
(Consider how the probability-matching Outcome could possibly have isn 
predicted from or explained by Guthrie's original theory.) One gee 
is quantification, By Sticking to the basic equations and derivations fro 


. Sree i < the 
them, Estes can make precise quantitative Predictions; he can also check t 
predictions against the data and test the y 


is a peculiar untestability to Guthrie's ow 


d n a yt con- 
It sometimes turns out that @ is ne 


stimuli. If a response persists, 
if the response changes, then the stimulus patterr 
short, Estes’ approach is by its very nature qu 
approach had been qualitative. 

The second advantage of Estes’ m 
into the theory are either 


In 
n must have changed. i 
antitative, whereas Guthrie ‘ 


ike P,) or easily estimated 
by finding the best-fitting 
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Guthrie's time one had to take a stand on the issue of mentalism versus 
mechanism, and Guthrie's stand was well over on the mechanistic side. But, 
by presenting a mathematical model that merely describes learning, Estes 
remains neutral on this issue. Both P, and 6 are obviously descriptive of 
the subject's behavior and nothing more. We see clearly, perhaps for the 
first time, that a mathematical model must necessarily be neutral on the 
issue of mentalism versus mechanism. For this reason mathematical repre- 
sentation of a learning experiment is perhaps the ideal way to avoid all the 
troublesome problems of mentalism and mechanism. 

A model, any kind of model, is supposed to represent the thing being 
, and no one expects it to represent the real 


modeled in some particular wa 
thing in all respects. If it did, it would be the real thing itself and not a 


model. For example, a model airplane is like the real thing in that it is the 
Same shape. It “looks like” the real thing, but upon closer inspection it is 
found to be quite different from the real thing. It is a lot smaller, and it 
is made of paper and balsa wood instead of aluminum. But these differences 
are acceptable; as long as the toy plane looks like a real plane it is an ac- 
ceptable model. The point of a model, however, is that, in being like the 
real thing in one respect, it may surprise us and give us “additional bene- 


fits” by being like the real thing in ways that we had not anticipated. For 
example, if we put a rubber band in our toy plane and wind it up, we may 
discover that it also flies. It may not fly in precisely the same way or make 
as much noise as the real plane, but nonetheless it flies, and this fact may 
be a very useful additional benefit. For example, if we put the model plane 
in a wind tunnel, we may discover that it shares a variety of aerodynamic 
Properties with the real airplane. In other words, the surprising similarities, 
the additional benefits, of a model may turn out to be very great benefits 
indeed. 

And so it is with a mathematical model of learning. We can work with 
the model with pencil and paper and derive predictions that could not, or 
would not, have been readily made from observing real subjects in real 
learning experiments. And this fact, too, may be a very great benefit. Estes 
(1958) also extended Guthrie’s analysis of motivation in terms of maintain- 
ing stimuli in a much more nearly complete and satisfying way than Guthrie 
had been able to achieve. Although Estes has recently gone on to other 
kinds of theoretical models, for many years he renewed interest in Guthrie's 
principles of learning. ee, A 

Another offshoot of Guthrie's contiguity theory is elicitation theory, as 
it is designated by M. Ray Denny and his colleagues. The first theoretical 
Statements (Denny & Adelman, 1955; Maatsch, 1954) were primarily con- 
cerned with an interpretation of extinction based on competing responses. 
In an experiment reported by Adelman and Maatsch (1955), two groups of 
rats were trained to run into a box for food, and then extinction was begun; 
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that is, food was withheld. 
top of the box to escape th 
other group was required t 
The back-up group stopped 


Out group continued to ru 
Adelman and M 


One group was permitted to jump unong i 
e presumably frustrating situation, BER 
o back up through the door and out of H mp- 
l running almost immediately, whereas the U 
n to the box and jump out of it for many U1 


; to 
3 j » animal learned 
aatsch argued that in each instance the animal | 

make that response in the 


out on one hand 
response is com 
back-up res 


alley that it had made in the goal sii aa ae 
and backing out on the other. Because the jump 
patible with running, 2 eralizes 
ponse is incompatible with running, and, because it P ian 
from the goal box to the alley, its occurrence in the alley effectively 
petes with the running response. 
Denny and Adelman (1955) 
in such a situation the 
answ 


e 
: - ‘cur, But th 
running continues to occur, I 


hy 
have raised the more basic question bie 
running response was acquired originally. ee 
er, which constitutes the fundamental premise of elicitation ii box. 
that approach behavior, or going forward, is elicited by food in the eee 
It is not that food is literally a reinforcer, a special kind of event that tate 
in an S-R connection, or that it provides a goal to which the an sire 
behavior is directed. Reinforcers are simply events that initially elicit ae 
ticular kinds of behavior like approach, 

to other stimuli in the situ 
been applied to a v 
electrical brain stin 


Approach then becomes ee S ee 
ation. This interpretation of Aor ior al 
ariety of phenomena, including the reinforcing effe 
ulation, by Glickman and Schiff (1967). 
Denny's elicitation theory 

sense that it is a contig 
have 


is related to Guthrie’s learning theory 1 
uity interpretation of aviors: 
no special properties other than that they elicit particular Belay a 
A second point of similarity is the emphasis given to the generalization 


š f: f 5 5 š Ror exam ale, 
a later response in a chain to earlier stimulus Situation, For exam] 
elicitation theory predicts that the extinction 


Maatsch would be quite different if the 
perceptually distinctive. If one 
both the persistence of runnin 
of running in the back-up 

Recently Denny has ex 
challenging problem of 


* p ich reinforc 
learning in which reinf 


results of Adelman on 
alley and the goal box had ay 
had been painted black and the other Rae 
g in the jump-out group and the rapid loss 
group should have been much tess marked. f 
tended elicitation theory to deal with the very 


avoidance learning (Denny, 1971). The basic COP- 
cept is that after some period of time following the termination of shock 
the rat will relax, and that once relaxation responses h 
tioned to a particul ental stimuli 


f the app 


ave become condi- 
the rat will approach 
aratus will appear to be i 
arning difficult in many avolc 
ior May generalize back to the 
8 away and, second, that running 
the apparatus and compete with 


ar set of environm 
such stimuli. In effect, the safe Part o; 


positive reinforcer. What makes avoidance le 
ance situations is, first, that relaxation behav 
danger situation and interfere with runnin 
away may generalize to the safe part of 
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relaxation. Avoidance performance has been shown to be influenced by the 
similarity between the danger area and the safe area and by the period of 
pi in which the animal is permitted to relax in the safe area (Denny, 
` 71). This ingenious interpretation of avoidance learning seems to have 
considerable viability, and, in Denny's hands, it can explain a variety of 
phenomena that have been extremely troublesome to other theorists. A 
major point of difference between elicitation theory and its Guthrian an- 
cestor is that Guthrie always emphasized the stimulus. Elicitation theory 
and requires little detailed analysis of the stimulus 
elements contributing to a given situation. Attention is directed to the ani- 
mall's response in a given situation because it is, after all, the response that 
is generalized from one situation to the next, and it is ultimately the re- 
Sponse that we must explain. 

In conclusion, we can observe that Guthrie's learning theory is built 
around one simple idea: Behavior is explained when the eliciting stimulus 
is found. When the stimulus cannot be found, then an appropriate stimulus, 
pe even class of stimuli, is postulated to exist. The intrinsic appeal of 
Guthrie's theory and perhaps the reason that it has endured, even if it has 
not been very popular, is its conceptual parsimony. The theory occupies a 


unique place in our narrative because it is clearly a logical extension, almost 
arlier theories of Pavlov and Watson, yet, 


ation, it provides forty years later the sub- 
like those of Estes and Denny. 


emphasizes the response 


a necessary extension, of the e 
subjected to only slight modific 
Sta Ae 

tance of contemporary positions 
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In contrast with our other theorists, Guthrie did not separate the experimental 
explication of his theory from its practical application. His theoretical writings 
draw heavily upon everyday experience and the casual observation of behavior. 
Thus, The Psychology of Learning (1935) and The Psychology of Human Conflict 
(1938) are easy reading and convey a sense of relevance. The main substance of the 
theory can be found in two short papers (Guthrie, 1930; Voeks, 1950). There is 
little biographical material on Guthrie, but in his last theoretical statement (Guthrie, 
1959) he provided some insight into his background and a rather personal last look 
at his own contribution. The best short introduction to mathematical learning 


theory is the chapter on this topic in Hilgard and Bower (1974). 


CHAPTER 
5 


TOLMAN 


career was spent at the University of Cali ornia at Berkeley; his writings 
and the mass of research that came out of the Berkeley laboratory made it 
the center of cognitive psychology for 
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i The major impetus to Tolman forming his view of behavior was the 
TisIng popularity of Watsonian behaviorism. The early behaviorists, Pavlov, 
Watson, and Guthrie, had all sought to dispel mentalism from psychology 
and to make psychology into a scientific discipline with objective methods. 
They admonished us not to speculate about what is going on in the mind 
of man or animal. The data of psychology were to be just the observations 
of behavior. Tolman was always sympathetic with this strategy, but he 
observed that ‘the early behaviorists had all confused the objective method 
of science with a mechanistic philosophy. One does not have to be a 
mechanist in order to study behavior, to be objective or deterministic, or 
to follow the precepts of science. 

The S-R association, which has been endorsed and advocated by almost 
all behaviorists except for Tolman, is in no sense an empirical fact; it is 
not an objective datum. It is an inference or conjecture about the physio- 
logical machinery that produces behavior. So, although Tolman could go 
along with the early behaviorists’ emphasis upon behavior, he saw the intro- 
duction of the mechanistic philosophy to explain behavior as unwarranted 
and undesirable. Behavior is no better or more scientifically explained by 
S-R connections than by thinking and willing. In either case a philosophical 
doctrine about the underlying reality n the systematic 
observation and analysis of behavior. The philosophy must even bias how 
we observe behavior. To be a behaviorist all one has to do is to look at 
behavior and to try to discern its lawfulness. (Tolman was thus a different 
kind of behaviorist. He believed in the elucidation of behavioral principles 
but rejected the mechanistic framework in which behaviorism had been 
cast. He rejected the S-R unit as a means of describing or explaining be- 
havior and rejected most of the other assumptions characteristic of the 


is superimposed upo 


Association ier: 
Ssociationistic approach. 


Tolman’s Perception of Behavior 


(1920, 1923) argued that the ob- 
directed toward some goal. 
d. Its behavior appears to 
an that the animal 


In his first theoretical papers Tolman 
servation of behavior reveals that it is generally 
The hungry rat directs its behavior toward foo 
have a purpose. This appearance of purpose does not me 
has a conscious purpose, a plan or intention, or that it 1s aware of its goal. 
It is simply to say that behavior itself appears to be focused on some end, 
like food. Because the animal's behavior looks as if the animal has a purpose 
in the more traditional sense of the word, Tolman coined a new term 
“purposive” to describe it. Literally it is @s if the animal had a purpose. 
“Purposive” thus describes behavior, whereas “purpose” is a hypothetical 
mental event. When we watch animals going about their daily business, 
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their behavior seem 

havior were g 

of behavior teleology. : as its flexibility. 
One of the chief things that Tolman saw in behavior was its ertain 

Behavior is not invariant. 

stimuli, which we call ‘ 


. be- 
A s is as if the 
S to be directed to certain outcomes. It f this view 
= p eo 
uided by the outcome, and Tolman often spoke 
as a kind of descriptive 


There are, to be sure, fixed reactions A and 
‘reflexive,” but they constitute a relatively he details 
an animal's total behavior. Most of the spe Ea made 
of behavior are unpredictable. We do not know what response Si ss: to use 
at the next instant, Behavior has an intrinsic flexibility or loose eliit ie 
Tolman’s expression, “behavior is docile.” The idea that stimuli qı that 
Sponses implies a kind of necessity, 

simply is not chara 
a Pavlovian experi 
For example, if w 
finds a tw 


uninteresting part of 


compulsion, or mechanical eg’ a 
cteristic of behavior except in certain saron zeneral: 
ment, that are not representative of behavior “i “oft and 
€ watch a bird building a nest we see that it flies iP 
ig, carries the twig back to the nest 3 
ferent ways until it is 


twig, comes back, and 


, and adjusts it in erei 
just right. Then it flies off and gorun a ene In 
adjusts it again in a somewhat ln iteneny it selects 
Picking out twigs the bird may sort through a half-dozen before flexible: 
One. All this time the bird’s behavior are app!” 
it does not look reflexive. The bird flies to a place where arene Aq) this 
priate twigs, then selects and carries the right twig back to the coe out 
time it is avoiding predators, flying around obstacles, and taking jy thing 
to eat and to defend its territory. Tolman (1923) argued that the R 
that seems fixed about instinctive behavior is the end product, the nen 
Nest, mating with the eating the food, or whatever. (In gente ent 
behaviors, Tolm end is fixed, 

Is v: 


appears to be extremely 


female, 
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of about a hundred different types of atoms.(To the associationist, the an 
is the atom out of which everything is built, But Toln a 
holistic., To continue the analogy, Tolm at we can have lil ‘i 
appreciation of the properties of water—its ess, fluidity, various physic 
and chemical properties—if we Tegar 


it si i sogen 
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Had Tolman merely stated his POint ‘of View, few Psychologists woul 
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have paid very much attention, but he did much more. He had a unique 
facility for translating his own individual perception of behavior into 
laboratory experiments, the interpretation of which seemed amenable only 
to his own cognitive kind of interpretation. Let us look at some of these 


remarkable experiments. 


Some Representative Experiments 
An interesting example of how Tolman’s different perception of behavior 
could be translated into new kinds of learning situations is a study reported 
1930). Macfarlane trained rats to swim 


by one of his students, Macfarlane ( 
lled with several inches of water, and 


in a maze. It was an ordinary maze fi 
the rats had to paddle their way past the several blind alleys to reach the 


goal box, where they could climb out of the water and obtain food. Granted 
that the animals learned to reach the food, there remained the question of 
whether they had learned something cognitive about the maze—how to get 
to the food—as Tolman claimed or had learned a series of stimulus-response 
connections as almost everyone else would have argued. To address this 
question, Macfarlane drained the water out of the apparatus, so that the 
maze was dry. Rats were put back into it so that they could run through it 
instead of swimming through it. Macfarlane found almost perfect transfer 
from swimming to running; that is, the animals were immediately able to 
run the maze with virtually no errors. Notice that completely different 
y are involved in swimming and running. 
The stimuli are different, too. The maze was dry instead of wet, it was warm 
instead of cold, and there was a floor underneath instead of a supporting 
body of water. It appears that what was learned in Macfarlane’s maze was 
NOt a series of movements in response CO specific stimuli but something much 
More abstract and holistic: the spatial layout of the maze itself. The rats had 


acquired a map of the maze. Tolman coined the phrase “cognitive map” 
as a label for the learned representation of the experimental situation. It is 
as if the rat learns where it is, where food is, where the blind alleys are, 
and where the open pathways are. Once this information has been assimi- 
lated into some kind of central map, the animal's task is simply to go from 
where it is to where the food is, and it need not do so by means of any 
fixed kind of locomotor system. If Macfarlane’s rats could have been out- 
fitted with wings, they presumably would have flown through the maze 
without making any errors. 

There is another series of studies, cond 
the “place-learning experiments.” The basic situation is a simple T maze, 
l. The rat starts at the south end of the maze and 


responses and even muscle systems 


ucted mostly in the 1940s, called 


as shown in Figure 5. 
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Train Test 
Fig. 5.1 Typical arrangement for a place-learning 
experiment, 
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if it has learned a place, it should still go east. Most of the oe e place 
experiments done at Berkeley have shown that the rat learns th 

(e.g., Tolman, Ritchie & Kalish, 1946). 
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availability of different kinds of stimuli. But there are some problems with 
a straightforward application of this S-R analysis. For one thing, when there 
are only intramaze stimuli, the animal approaches them or makes a right 
turn in their presence, both simple S-R associations from the S-R point of 
view. But these simple associations are apparently not formed when there 
Vhy does the rat use the information pro- 


hen it can use cues close at hand? 
d by Kendler and 


are extramaze cues. Why not? V 
vided by distant extramaze cues w 
A further complication in place learning was note 
Gasser (1948). They found that if rats were given fewer than twenty trials 
on the original T maze, they showed place-learning in the test situa- 
Hon. But if they were given more than twenty trials they learned the 
response. Again we may conclude that, although the rat can and sometimes 
does learn the response, Tolman was apparently right in emphasizing the 
portance of place learning. The rat's initial and preferred mode of re- 


Sponding is place-wise. Kendler’s results indicate, too, that once the rat 
there is further learning that resembles 
The behavior becomes auto- 
he stimuli that con- 


has learned to solve the problem, 
the old-fashioned idea of habit acquisition. 
Matic in the sense that it no longer depends upon t 
trolled it originally. 

Tolman denied that reinforcement is necessary for learning. Behavior 
changes, Tolman admitted, when there is food in the apparatus, but this 
change should not be taken to mean that the food causes learning. In 
Tolman’s view, the animal learns about the layout of the maze, where the 
blind alleys are, and so on, simply by running around in it and exploring it. 
Learning consists of the incorporation of information about the environ- 
Ment, Once this information is incorporated, the animal will use it to obtain 
To test this view of learning, Tolman and his students 
e become known as the “latent- 
’ In a typical experiment (Tolman & Honzik, 1930b) 
ained in a complex maze. One group was run 
under normal conditions, with food always available in the goal box. The 
second group was run with no food in the apparatus. The critical third 
group had no food at first, but food was introduced on the eleventh day. 
The results for each group are shown in Figure 5.2. Performance was poor 
when there was no food present; little learning was evident. (There is some 
question about whether these groups showed some gradual improvement in 
error reduction.) But when food was introduced in the critical group per- 
formance improved dramatically. The rapid reduction in the number of 
errors in the next trial or two brought this group up to the level of the 
group that had received food all along. 

Tolman and Honzik suggested on the basis of these results that the 
learning was actually identical for all three groups but that, for the non- 
reinforced animals, the learning was “latent.” Learning was not manifest in 
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Fig. 52 Results of the Tolman and Honzik (1930b) 
latent learning experiment. Control group NR re- 
ceived no reward in the maze; control group R 
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die changes in performance are not synonymous with learning; learning is 
something else. 

Let us consider another experiment reported by Tolman and Honzik 
(19309. Rats were trained in the apparatus diagramed in Figure 5.3. There 
Were three paths from the start box to the goal box. The straight path A 
was the shortest and most preferred route. The animals were given a num- 
ET TAN they were forced to take paths B and C, somewhat 


longer and less direct routes to the goal. After this experience and after a 
preference had been developed for path A, the path was blocked at point X. 
the animals 


The question that concerned Tolman and Honzik was how 
would detour around this block. The preferred detour was path B, the next 
shortest path. But then a block was imposed at point Y, requiring animals 
quite near the goal to run all the way back to the starting point to take 
another detour. The question then was how accurate was the animals’ 
cognitive map. Would they again take detour B, or would they choose the 
only effective detour, path C? They did not take the next most preferred 


route, path B, but apparently had the “insight” to take the only path lead- 
t roundabout route, path G: 


Ing to the goal, the least preferred and mos Ee 
A common assumption underlying most S-R associationist accounts of 
behavior is that the §-R association is built up gradually and automatically. 


food 


path A 


path C 


start 
Fig. 5.3 Apparatus used by Tolman and Honzik 
(1930c) to demonstrate “insight” in rats. (Originally 
published by the University of California Press: 
reprinted by permission of The Regents of the Uni- 


versity of California.) 
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to test hypotheses by the acquisition of incidental information; a direct test 


appeared to be necessary. For example, let us suppose that the situation is 
arranged so that going left is correct, and let us suppose that a particular 
animal starts by testing the black hypothesis. What will happen is that 
going to the black stimulus will be correct when it is on the left but not 
when it is on the right. The animal ought to be able to learn something 
incidently about the correctness of the left by testing the black hypothesis, 
but evidently it does not. 

In a dramatic test of the noncontinuity of discrimination learning, 
Krechevsky (1938) found that it was even possible to change which 
Stimulus was correct, and if this change was accomplished fairly early 
in the course of discrimination training it made no difference in how 
quickly a group of rats would ultimately learn the discrimination. In other 
words, Krechevsky has shown in another way that nothing was learned 
about the correct cue while the rat was testing an irrelevant hypothesis. 
Other experimenters (for example, Spence, 1945) rose to the challenge and 
found results contradictory to those of Krechevsky, and for many years the 
question went back and forth. Indeed, it is still not clear what conditions 
lead to continuous learning of the sort that Spence and other S-R theorists 
require, and what conditions lead to the noncontinuous discrimination 
learning that Krechevsky had discovered. But, however the issue is finally 
resolved, it seems safe to conclude that there are at least some conditions 
under which discrimination learning is an extremely active process and does 
not consist simply of gradually strengthening S-R associations. The rat can 
direct its behavior toward particular stimuli to test particular hypotheses. 


Tolman’s Theory of Learning 


We have described some parts and pieces of Tolman’s theory of learning. 
Let us now see if we can put these pieces together into a coherent picture. 
‘Tolman started from the premise that all behavior is directed toward some 
goal, that the rat has a “demand” for some object such as food, a mate, 


safety, or whatever. In the laborator arranged so that 


the rat is hungry and the goal is obt 


about intentionally by the experimenter 
the same purposiveness of behavior is apparent when the animal’s goals are 
less insistent. In short, we work with animals that are highly motivated, 
and we provide a goal object because this guarantees that the animal's 
behavior will be directed toward the goal that we provide. 

In instrumental-learning situations we arrange that the animal must pro- 
duce some particular behavior. It must run a certain path, press a certain 
bar, or make some other specific response tO obtain food. As we order the 


y the situation is often 
aining food. This situation is brought 
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It runs toward that place. If we impose a particular task that deca of the 
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task to produce food: It presses the bar or whatever, What learning 
sists of, then, is the expectancy 


icula! 
> 5 ave a particu 
that a particular event will have a p 
consequence. The 


pee The 
most general formula for an expectancy is pat a 
animal expects that in the Presence of a particular sign (S) a pal 
behavior (R) will have a particular consequence (S*) 
In Tolman’s own experiments inf 
ordered Spatially; his rats h 
was, how it could be re 


ent was 
ermation about the environm 
ad to learn 5 


ached, and so on. 
tion in time, For example, in a P 


sign for the US: The bell becomes 
the bel] arouses an expec tancy of food. Th ; lus 

Tolman saw stimuli as serving Several different functions, A stimu : 
can arouse an expec tancy. It can convey, in effect, that, if the animal oe 
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tion a cue can signify that food is down at the other end of the alley, oF } 
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put the rat in a T maze, and it has to turn left in order to obtain food. 
That is how we perceive the experimental arrangement. What Tolman was 
Saying is that that is how the rat perceives it too. The rat is simply perceiv- 
ing the regularities in the experimental world that we create for it, and it 
responds on the basis of its demand for a certain goal object such as food, 
again, because we have deprived it of food. 


Motivation 


Tolman was one of the first psychologists to treat the problem of motiva- 
tion in a systematic manner. He proposed that an animal is stirred to action 
when it has a “demand” for a particular goal object. Demand was assumed 
to be governed by two kinds of factors: deprivation and what we may call 
incentive. One of the most interesting and dramatic demonstrations of 
incentive motivation was reported in an early experiment by Tinklepaugh 
(1928). Tinklepaugh trained monkeys to work at a discrimination task. 
While the monkey was watching, Tinklepaugh placed a small piece of 
highly preferred food like banana under one of two containers; then after 
a brief delay the monkey was allowed to approach and pick up the con- 
tainer in which it had seen the food placed. Tinklepaugh’s animals quickly 
learned to play this game. Then from time to time Tinklepaugh would trick 
them, The container would be loaded with the highly preferred food, and 
surreptitiously an acceptable but less preferred food such as a piece of 
lettuce would be substituted. On these occasions the monkey would typically 
refuse the food and start looking carefully all around inside and outside 


the container, It would look on the floor and around the room. Occasionally, 
it would become upset enough to shriek at the people in the room. It is clear 
that the normally cooperative subject was disturbed both behaviorally and 
emotionally. 

According to Tolman’s interpretation, the monkey had an expectancy 
not just of food but of a particular kind of food. The results of switching 
food objects show that the animal's behavior is governed not by the im- 
mediate consequences of its acts but by what it expects Its behavior to 


Produce. 


The importance of an expectancy for a particular food object was also 


demonstrated in Tolman's laboratory by Elliott (1928). Elliott trained 
different groups of rats to run in a maze for food. For one group the food 
was sunflower seeds, for another group bran mash. The performance of the 
two groups, shown in Figure 5.4, indicates that bran mash was much pre- 
ferred. On the tenth day Elliott shifted incentives for the mash group. When 
the animals were switched from the more preferred to the less preferred 
food, they showed a sharp deterioration of performance. There was even 
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Fig. 54 Results of the Elliott (1928) study in which 
rats switched from mash (preferred) to sunflower 
seed (less preferred) performed more poorly than rats 
rewarded all along with the seeds. (Originally pub- 
lished by the University of California Press; re- 


printed by permission of The Regents of the Uni- 
versity of California.) 
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Fig. 5.5 Results of the Tolman and Honzik (1930a) 
experiment in which groups of rats were either 
hungry (H), Rewarded (R), neither, or both (HR). 
The animals were run one trial a day in a I4-unit 
maze. (Originally published by the University of 
California Press; reprinted by permission of The 
Regents of the University of California.) 
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syntactical principle: Learning occurs when an S and an R become con- 
nected. A rat responds to a particular situation because an appropriate S-R 
connection has been strengthened or reinforced. In other words, all our 
theorists have had a single kind of syntactical unit that has been assumed 
to be very simple and universal. But Tolman said, on the contrary, that 
behavior is determined by a much looser and more flexible kind of system. 
The detour problem, for example, is solved not by an appropriate response 
to a particular stimulus but by a process like looking at a map and seeing 
what route to the goal remains open. However we may conceptualize the 
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t as reading its map, this process 15 intrinsically much more complex 
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about how maps are constructed and how they are read. T o oe 
emphasized the flexibility and richness of behavior; he also fe eeil 
in which these aspects are produced by the complexity of mg zi ie 
rules. These processes, which are very like perception, are not ey Ta of 
stand. The best way to understand them is simply to run a a ci 
experiments to find out how they work. What kinds of sae K a 
have? How do they perceive the world around them? What anes 95 coe 
lems can they solve? At one level Tolman’s principles of behavior ate on 
simple: The animal acts in a certain way because it expects E o a 
havior will have certain consequences and because simultaneously 
some demand for those consequences. . a eE 

Much of the language with which Tolman described behavior, s neat 
tainly much of the language in his theory, was frankly mentalistic. aa d 
on that account, Tolman has been widely criticized as unscientific, un 
havioristic, and subjective. Guthrie, for example, made this criticism. we 
the criticism is unfounded. We have seen that such essential terms in pa 
man’s theory as “expectancy” and “demand” are defined behaviorally. ted 
know an expectancy from the fact that the animal's behavior is ia i 
when the conditions of the experiment are changed, as in Tinkelpaug = 
study. We know that the animal has a demand for a goal object by j 
vigor with which it runs to the goal. We should thus not be deceived 7 
the kind of language Tolman used. It has to be viewed in historical P 
spective; it was no doubt introduced as a kind of antidote to the heavily 
mechanistic language of Watson. Actually, we may ask of the carly Be 
haviorists, like Watson, whether their concepts and language were any more 
scientific or any more objective. Is it any better to speak of behavior in 
terms of reflexes and inferred neural machinery? Tolman’s use of mer 
talistic language merely dramatized his claim that a purely mechanical 
model cannot adequately explain animal behavior. 

In the 1930s psychologists became much more sophisticated in their con- 
ceptions of the relationship between data and theory. We have noted 4 
previous chapters the tendency of earlier theorists to regard behaviors 
phenomena as direct expressions of underlying mechanisms. The observed 
correlation between a stimulus and a response was assumed to reflect, some 
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meant the increased probability of a response in the experimental situation 
ee a stronger neural connection. Tolman recognized the importance of the 
distinction and tried to make it explicit. He has told us (Tolman, 1936) 
that his empirical terms are all objective data and that his theoretical terms 
are “intervening variables.” 

Tolman’s theory was therefore quite different from those of his prede- 
cessors in that he made no pretense that his explanatory terms, intervening 
variables such as demand and expectancy, have any reality or can be located 
somewhere inside the organism. The intervening variable is essentially an 
avior itself. It is much like the mathematical con- 
struct discussed in Chapter 4. Intervening variables are so called because 
sized to intervene conceptually between stimulus input and 
behavior output, not because they are assumed to be located somewhere in 
the animal. Expectancy is assumed to be a determinant of behavior in any 
simple situation, as when the rat runs in an alley for food, but it is not 
clearly manifest in behavior except in experiments, like those of Tinkel- 
paugh, that are especially contrived to reveal it. Demand is assumed to 
operate in many situations but is most clearly observed in a latent-learning 
experiment. The intervening variable is in turn assumed to be dependent 
upon antecedent conditions like environmental information in the instance 
of expectancy and deprivation in that of demand. But this dependence is 
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information. Its response depends largely upon the ingenuity of the experi- 
menter in demonstrating the assumed relationships between observable 
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Tolman’s Impa 
Tolman was an eclectic theorist. He gathered id z 
and often did not combine them very systematically. In a number of dif- 
ferent theoretical statements spanning a quarter-century he emphasized this, 
that, or another aspect of behavior but did little systematic organizing of 
his theory after 1932. Over the years his emphasis changed, and the language 
he used to describe his unique perception of behavior often changed dra- 
matically. Tolman was also eclectic about the kinds of laws that he sought 

he emphasized the capacity 


to incorporate within his theory. At one time 
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of animals for particular kinds of learning or for processing certain sorts of 
information (Tolman, 1932). At other times he observed that behavior might 
have an intrinsic probabilistic character because the world in which animals 
live is not itself completely predictable (Tolman & Brunswik, 1935). But 
Tolman was steadfast in emphasizing the purposiveness and docility of 
behavior, and he always emphasized that what is learned is an expectancy 
or the value of signs, rather than $-R connections. 

In short, Tolman was too open to new ideas and not systematic enough 
in integrating them to build a precise, formal theory. His unique and 
enduring contribution was primarily to relate his perception of the behavior 
of animals, rather than in any systematic statement of learning principles. 

The success of Tolman’s theory was due in large part to his own efforts, 
specifically, to his theoretical papers of the 1930s and 1940s and to the mass 
of provocative research reported around 1930. As Tolman himself became 
less active, about 1950, there seemed to be few students or disciples to carry 
his work forward. Others have, however, made some important extensions 
of his learning theory. For example, Osgood (1950) showed how easily an 
expectancy theory can be applied to the difficult problem of avoidance 
learning. In Chapter 10 we will see that there is renewed interest in the 
expectancy concept at the present time just because it provides a powerful 
approach to troublesome phenomena like avoidance. Rotter (1954) developed 
a quite influential social-learning theory in which Tolman’s basic concepts 
were extended to the case of human motivation in everyday kinds of situa- 
tions. This development has continued in a number of different directions, 
so that the conception that behavior is a joint function of something like 
expectancy and something like demand has become a central part of many 
contemporary theories of human motivation, 


It should be obvious that Tolman stood outside the mainstream of 
behavioristic psychology. And perhaps his greatest contribution has been his 
indirect influence on those who have been in the mainstream, Initially he 
stood in opposition to Watson's simple S-R connectionism and mechanistic 
philosophy. In later years his chief opponent was Hull, and it is a great 
tribute to Tolman that Hull was forced to hypothesize some very sophisti- 
cated S-R devices to explain such phenomena as | 
learning. Hull also had to incorporate motivational concepts within his 
system. In other words, the complexity and sophistication of Hull's system 
were largely a response to the challenge posed by Tolman. And that, ae we 
shall see, was a very large contribution indeed, 


atent learning and place 


After Tolman had his say it became impossible for anyone to take seri- 
ously a simple mechanistic S-R interpretation of behavior 
possible to ignore motivational principles in any complete account of be- 
havior, and it became impossible to ignore the great flexibility, adaptive- 


ness, and apparent intelligence of animal behavior, 


it become im- 
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ave substance to Tolman’s theory is described in 
Purposive Behavior in Animals and Men (Tolman, 1932). The main features of the 
theory laid down at that time remained relatively unchanged, but the details of the 
theory and the language Tolman used to describe it varied considerably over the 
years. Tolman wrote a minor book on motivation, Drives toward War (Tolman, 
1942), and a number of theoretical papers that have been reprinted in his Collected 
Papers in Psychology (Tolman, 1951). The latter are his most readable and perhaps 
Tolman wrote with a lot of spirit and wit but not much 
al precision, which makes his work entertaining but hard to 
d Meehl (1954) tried to systematize the theory, but the 
Tolman’s last papers include a retrospective view 
and a short autobiography (Tolman, 1952). 


The early research that g 


most important works. 
systematic style or form: 
grasp. MacCorquodale an 
results were not very satisfactory. 
of his life's work (Tolman, 1959) 


CHAPTER 
6 


HULL 


Clark L. Hull (1884-1952) was able to synthesize into a unified system 
many of the achievements of earlier theorists ‘He did, in fact, stand on the 


s, building on their strengths and avoiding some 


em of behavior was in the S-R tradition, but it 
was deductive and mathematical in form, and i 


havioristic, with hardly a taint of mentali 
was explicit and highly testable, 
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founded an empire with Yale University as its capital. More than anything 
else, Hull generated research. 

Although Hull was nearly the same age as Tolman, his commitment to 
the study of learning came approximately a decade later. During the inter- 
vening years Hull had distinguished himself by his research in such diverse 
areas as concept formation (Hull, 1920) and aptitude testing (Hull, 1928). 
In 1929 he went to Yale to head a group at the Institute of Human Rela- 
tions, whose mission was to study the place of learning in the conduct of 
human affairs. Hull's original orientation was strongly Pavlovian. He was 
evidently impressed with the systematic nature and scientific aura of Pavlov’s 
work, and he depended almost exclusively upon the principles of Pavlovian 
conditioning to account for learning. Of course, he was not alone in this 
commitment; most theoretical statements in the early 1930s were made 
within a conditioning framework. 

Hull's first papers on learning theory in 1929 and 1930 were attempts to 
show that purposiveness in behavior could be explained with Pavlovian S-R 
associations, In other words, Hull sought to extend the Pavlovian frame- 
work from the original conditioning situation to the kind of situation in 
which behavior appears highly flexible, adaptive, and intelligent. Hull did 
not deny these descriptive attributes of behavior. He sought rather to derive 
them from simple conditioning principles. The argument was like that made 
a few years later by Guthrie (see Chapter 4). Hull argued that there is a 
tendency for the consummatory response to be elicited not only by goal-box 
stimuli but also by stimuli similar to those in the goal box that arise in 
other parts of the apparatus. ; 

Then there was an important new development. Thorndike had returned 
to the study of learning and was vigorously defending the idea that learning 
occurs as a result of reinforcement. By the mid-1930s a confrontation between 


Thorndike’s reinforcement position and the better-established conditioning 
Hull apparently came when he 


position seemed inevitable. The crisis for y 
wrote a long analytical review of a new book by Thorndike (Hull, 1935). 


In this review Hull noted a major inadequacy of conditioning theory: its 


failure to deal convincingly with the phenomena of motivation. Hull saw 
that motivation may be viewed as either a learned aspect of behavior (as 


Guthrie regarded it) or a behavioral determinant quite independent of 
learning (as Tolman regarded it), but one way or the other, it had to be 
given more status than it was afforded at that time. 

A final factor that gave form to Hull's theory came not from the intel- 
lectual environment but from the man himself. He was greatly impressed 
by the elegance and power of quantitative and deductive methods in science. 
In his early theoretical papers Hull proclaimed that the proper strategy for 
science would be to start with certain specific, testable postulates even if 
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they would have to be based upon minimal evidence. Concrete, empirically 
verifiable deductions could then be derived from these postulates. When 
these deductions were tested, the system of postulates would then be either 
confirmed or shown to require modification. The task of the theorist would 
therefore be to formulate postulates in such a way that they would lead to 
unequivocal deductions. If there were no question about what inferences 
the theory led to, the deductions from the theory could be tested by anyone 
who cared to test them. The worth of a theory must then ultimately reside 
in how much research it generates and how consistent with its theoretical 
deductions the resulting findings are. We may note that applying this 
Strategy to predictions about behavior necessarily requires us to be be- 
haviorists. We cannot be concerned about the mental or physiological events 
giving rise to behavior if the postulates themselves are framed in behavioral 
form, as they were in Hull's work. 

To summarize the several factors that formed the conceptual background 
for Hull's theoretical synthesis, first was the challenge presented by Tol- 
man, both by his view of the purposiveness and goal-directedness of Der 
havior and by his emphasis upon motivation and the important part that It 
plays in behavior, Second was Thorndike’s concept of reinforcement, which 
came to the fore to challenge classical conditioning as the universal learning 
process. The third factor was Hull's desire to create a quantitative and 
deductive system to put behavior theory on a strong scientific footing. Hull 
struggled for years to establish a postulate system that he thought would 
account for what was then known about learning and motivation. When 
the final system was presented in his major book, Principles of Behavior 
(Hull, 1943), it embodied all the characteristics that we have described. It 
was a system of intervening variables, most of which were put in mathe- 
matical form. It incorporated distinct motivational and learning mecha- 
nisms, and learning was said to be based on reinforcement, rather than just 
on contiguity. Let us look at this 1943 theory. 


Hull’s Theory 


(The ultimate function of behavior in an animal, according to Hull, is to 
enable it to solve its biological problems. } -onsider an animal that has a 
need. Perhaps it has a need for food. A uséful reaction to such a problem 
would be for the animal to become active. It does not matter too much 
what kind of behavior the animal engages in; as long as it does something, 
it is likely to improve its chance of survival. Let us suppose further that 
while the animal is engaged in this increased activity it accidently makes a 
response that leads to food; the food eliminates the original need and solves 
the animal’s immediate biological problem. What an elegant system we 
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would have if this event, the reduction in need, were to serve as reinforce- 
ment and produce learning of the lucky response! The animal would then 
be, in effect, an automatic problem-solving system. Need would produce 
behavior, and the particular behavior that e ae need would be 
gradually learned. Animals would thus come to adapt to the special require- 
ments of their environments in solving their particular problems. This is 
precisely the kind of system that Hull postulated. 

First, there is drive. The animal’s need state, whether it be hunger, thirst, 
sexual arousal, pain, or some other type of biological problem, (produces a 
state of motivation that Hull called “drive.” Drive activates and generates 
avior, just behavior. Second, there is( reinforce- 
ive is reduced so that there will be learning 
animal's problem. Over a number of trials, 
ne increasingly efficient and more highly 
animal will become increasingly adept at 


behavior<no particular beh 
ccurs whenever dr 


ment, which 
of whatever response solves the 
the animal's behavior will becor 
adapted to its environment. The 
solving its problems in a given environment. — 

The same mechanism works in the laboratory. An animal in a box is 
suddenly in pain because of an electric shock applied to the grid floor. The 
animal displays a lot of behavior; it scrambles around and jumps and cries. 
In the course of this disorganized behavior it happens to press the bar in 
the box and terminates the shock. How elegant it would be if the bar-press 
strengthened; then the next time the rat was 


response were automatically 
it would be somewhat more likely to 


in such a difficulty (on the next trial) 5 a 
press the bar again. (Hull's scheme predicts that animals will learn to solve 
s both in the laboratory and in nature. Hull's system provides 
ated motivation, drive, and S-R learning produced 
as called “habit.” Let us see how 


their problem 
a basic concept of need-rel 
by reinforcement. The S-R connection w 
ere developed in more detail.) 


(Aull (1943) postulated that drive and habit multiply together to deter- 
mine the strength of behavior. This proposal was based upon a pair of 
experiments conducted by two students, Williams (1938) and Perin (1942). 
Williams trained groups of rats to press a bar for food, each group being 
given some fixed number of reinforcements ranging from five to ninety. 
juisition the response was extinguished in all animals. 
nction that Williams found in each group is shown 
in Figure 6.1. Perin’s experiment was conducted in exactly the same way. 
Different groups of animals were given a fixed number of reinforcements for 
pressing the bar and then extinguished. Perin’s results are also shown in 
Figure 6.1. The major difference between the two experiments, and pre- 
sumably the reason the results came out differently, is that Williams’ animals 
had been food-deprived for twenty-two hours at the time of testing, whereas 
Perin’s animals had been deprived for only three hours. Hull noted that the 
strength of behavior depends upon both the animal's motivating conditions 


these basic concepts W 
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Fig. 6.1 The famous Perin-Williams data showing 
how the strength of behavior, as measured by re- 
sistance to extinction, depends on both current mo- 
tivation conditions (drive) and prior learning (habit). 
(From Perin, Behavioral potentiality as a joint func- 
tion of the amount of training and the degree of 
hunger at the time of extinction. Journal of Experi- 
mental Psychology, 1942, 30, 93-173, Copyright 1942 
by the American Psychological Association. Re- 
printed by permission.) 


at the time of testing and the amount of prior learning, He analyzed the 
data further and found that it was possible to fit a mathematical learning 
curve to the results of each experiment. In each case th 


e equation came out 
in this form: 


Behavior strength = Ad 10-?*) 


When 4 indicates the level of performance ultim 
deprivation condition, B is the “growth constan 
the habit is formed, or how fast the anim 
performance, and N is the number of 
equations were obtained for Perin’s and for Williams’ 


that the term B was almost identical in the two experiments; the only real 


in the constant A. Hull therefore 
1 This equation describes the growth over time of a great v: 


It is likely to be applicable to any system on which there is some limit to how much 
growth can occur. It is based on the assumption that the Tate of growth is roportional 
to the amount of growth still possible. prop 


ariety of biological systems. 
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tentatively identified the factor (1 — 10-25) with habit and emphasized that 
it was not dependent upon deprivation; it seemed to depend only upon the 
number of prior reinforcements. Similarly, Hull identified the constant A 
with the animal's motivational state, or level of drive, which seemed to 
depend only upon deprivation conditions. The strength of behavior then 
becomes motivation times learning, or drive times habit, or D x H. 

Here we have a mathematical equation, an approach toward quantifica- 
tion, that describes behavior as a function of the two factors that it is sup- 
posed to depend upon, motivation and learning. We note, too, that, accord- 
(and in accordance with common sense) neither mo- 


ing to this equation 
itself will tell us how much behavior we will 


tivation nor prior learning by 
obtain from an animal. Thus, if we have an animal running rather slowly 
in an alley, we cannot tell whether it is not motivated but has a well- 
learned habit or is highly motivated and is just beginning to learn. The 
same response strength can be produced in either way. Consequently it is 
assess either drive or habit from a single observation. It takes 
1s, together with some assumption, such as theD x H 
what is determining the behavior in a particular 


impossible to 
a series of observatior 
equation, to establish 


instance. 
The principles of quantification that appear in Hull's theory were rather 


elaborate and, as it turned out, rather prematurely formulated. Hence we 
do not have to be concerned with his mathematical formulations. But it is 
important to understand the kind of deductive system that Hull attempted 
to create, According to Hull, science depends upon systematic observa- 
tion and measurement, but in addition science requires that sooner or later 
there be basic laws, preferably in mathematical form, from which it is 
possible to deduce theoretically what the results of experimental observa- 
tions should be. Let us digress briefly to see how this deductive process 
works in the well-established science of classical mechanics. 

In the seventeenth century Isaac Newton observed that if a pendulum 
and forth, it moved with a fixed period; it was therefore 
fast a particular pendulum swung. Newton did 
a number of experiments to see what determines this fixed period, and some 
of these experiments were rather interesting. If a heavy weight was put on 

isely the same speed; weight was not a 


the pendulum, it swung at preci n 
determining factor (which is rather interesting because the results are 
counter to intuition, or at least to some people's intuition), The amplitude 


of the motion—how far the pendulum swung—did not affect the period 
either, which is also a rather interesting negative result. But Newton dis- 
covered that the period of the pendulum did vary with its length. In fact, 
length turns out to be the only variable that affects how fast the pendulum 
swings. Having established some facts about the simple pendulum, Newton 
proceeded to derive an equation to describe his results. He assumed that the 


was swung back 
possible to determine how 
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force of gravity always operates downw. 


ard, and with a little mathematical 
manipulation he was able to conclude tl 


hat the acceleration of the pendulum 


isa = a when F is the force of gravity 
m 


and m the mass of the pendulum. 
Then, happily, when this e 
m term drops out, so th 
that the mass of the pendulum is not a f 
Indeed, all of Newton's observati 
rewrote the equation as F = 

postulate that summarized per 
matical models 


quation is solved to determine the period, the 
at this deduction is consistent with the observation 
actor in determining its period. 
Ons were consistent with the equation, He 
m X a, and in this form it was the basic 
ndular motion, But, as with many mathe- 
» there were a few additional benefits. The F = mxa 
equation turned out to describe not only the behavior of pendulums but 
also the behavior of cannon balls, falling stones, celestial bodies, and the 
motion of the earth about the sun, Observation of an immense variety of 
phenomena revealed that Newton's equation was almost universally valid. 


Now we know of situations in which Newton's equation fails to describe 
scientific observations, but for more than two ce 


mental challenges, 
Hull attem 


nturies it met all experi- 


pted to follow the Newton 
simple experiments like the 
these results in terms of 


ian model. He started with a few 
studies by Perin and Williams 


and summarized 
a simple equation: 


Behavior Strength = D x H 
This equation then sery 


es as a postulate or tentatiy, 
the determinants of behavi 


or in general. Its generality is tested by means 
of a variety of Experiments the outcomes of which are deduced from the 
postulate. If the deductions are consistent with observation, well and good; 
the postulates have gained in generality. If the results are inconsistent with 
the postulates, then the postulates must be modified, In what follows we 
will examine Some of Hull's postulates, 


e hypothesis about 
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Hull (1943) 
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l lated d Perties to drive, We have 
2 = rela iv als hf < 
KERAY s hat he related driv ; al’s biological 
sidering it an Immediate, unlear 


example, a hungry ani 
per se; it is just motivated. T) 
sources of drive (Brown, 1961), but in each 


motivation, that is, drive, is produced. Differ, 
multiply the existing habit 


lavior, For 
about its hunger 
‘ arded as different 
Mstance the same kind of 
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in any particular behavior. One prediction from this postulate is that, after 
as learned a particular response, such as pressing a bar, 
the execution of this response should not be dependent upon the animal 
being hungry. If the rat were suddenly shifted from hunger to thirst, the 
behavior ought to persist because it would be motivated by the new source 
of drive. Of course, extinction would occur unless the source of reinforce- 
ment were shifted at the same time from food to water, but the point is that 
there should be no sudden loss of behavior when the animal is first mo- 
tivated inappropriately. Some early experiments tended to confirm this 


remarkable prediction (for example, Webb, 1949), but a number of more 
recent experiments have failed to find such motivation transfer when proper 
f drive that can really be manipulated 


care has been taken to use sources 0 
independently. Food and water deprivation are inappropriate, it turns out, 
because when an animal is deprived of water it is both thirsty and hungry 


(Grice & Davis, 1957). For many years Judson Brown was the chief spokesman 
for the idea that different sources of drive are interchangeable, but recently 
hunger-fear conflict situation, the behaviors mo- 
tivated by hunger and by fear appear to be motivated relatively independ- 


ently (Brown, Anderson & Brown, 1966). 
implication of Hull's concept of generalized drive: An 


ibute an additional increment of 
strength of behavior. For example, a hungry rat 
a bar for food if it is also a little thirsty, a 
little frightened, or has some other source of irrelevant drive. Again, some 
early experimental reports suggested that this was the case (for example, 
Perin, 1942), but more recent and careful experimentation has shown that 
the summation of different sources of drive is very unpredictable. The effect 
can be found with some sources of drive and in some situations, but it 
appears to have little generality (this literature is reviewed by Bolles, 1975). 
Evidently, the motivation of behavior cannot be attributed to a completely 


general kind of mechanism, as Hull originally postulated. 
Of course, Hull knew that animals make specific responses when satis- 
fying particular needs. As early as 1933 he had shown that the rat can learn 


to make one response tO obtain water when it is thirs and another response 
to obtain food when it is hungry. But in formulating | his principles of 
behavior Hull retained the notion of drive as a generalized energizer and 
added an additional postulate. that specific stimuli (equivalent to what 
Guthrie called maintaining stimuli) are characteristic of each need state. 
These stimuli were said to have no motivational function: they were simply 
stimuli to which adaptive behavior, like behavior to satisfy the need, could 
be conditioned. This postulate therefore contained virtually the entire sub- 
stance of Guthrie’s motivational principles. but H 
of the other hypothetical properties of drive. 


a hungry animal h 


he has reported that in a I 


‘There is another 
irrelevant source of drive should contr 
drive, leading to greater 
should be more likely to press 


ull had, in addition, all 
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Learning 


The basic construct in 
Whereas he assumed dr ifcityol 
behavior, he proposed that habit was very specific{In fact, all the specihe an 
behavior was attri “he emphasized the point by placing = a 
scripts S$ and R around the symbol H. Thus sHp indicates the tendena ben 

i se. In his 1943 postulate syster 
of the number of reinforcements (see 
the amount and delay of reinforcement. pat 
to press a bar more quickly if it has eee 
a large, immediate food pellet. But in 1952, when Hull revised a an f 
the postulates, he said that (habit depends only upon the number o 
reinforcements, >) v 
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abit, according to — 
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generalized and unable to dire 
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example, the rat should learn 
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Hull and his collaborators 
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ae side that reduced drive. Drive reduction 
learning of the new response. These results 

therefore Provided nice confirmati 's Postulate, 
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The rat typically does not maintain its body weight, and even when the 


procedure does work it is not certain that there are not sensations in the 
head that accompany the passage of food on its way to the stomach, Thus it 
appears that some kind of mouth or head stimulation may be a much more 
important factor in reinforcement than placement of food in the stomach 
and the reduction in drive that it produces. 

Another problem connected with the drive-reduction hypothesis is the 
discovery by Olds and Milner (1954) that electrical stimulation of certain 
areas of the brain is reinforcing. Since the initial discovery of this phe- 
nomenon there have been many studies that show learning reinforced by 
brain stimulation in a variety of situations. In some cases this source of 
reinforcement is enormously effective and preferred by the rat to eating, 
mating, or any other kind of more natural consummatory response. But, if 
learning occurs under these conditions, then we have to ask where is the 
Source of drive that is being reduced? Where is the “need” for the stimula- 
tion? There are also a number of more natural situations in which learning 
Occurs in the absence of an apparent need and in which there seems to be 
No possible drive reduction. For example, it was discovered that the rat 

a complex situation which 


would learn a maze problem in order to get into 
it could explore. Exploration thus seems to be a kind of consummatory 


response the occurrence of which is reinforcing. Again we may ask where 
is the drive that is reduced? Some writers have gone so far as to invent a 
hew source of drive, claiming that the rat becomes bored in a familiar situa- 
tion and that exploration reduces this new source of drive. But again we 
may ask what is the underlying need that produces boredom? How does 
boredom threaten the biological integrity of the animal? It can be argued 
that animals have to explore their natural environments if they are to sur- 
vive, so there is a need in some sense, but this long-term and subtle need is 
very different from the brutal necessity of having food or water. 


The same 
argument could of course be made about mating as a source of drive. It, too, 
is a need in a biological sense, but it is not the kind of need and does not 
have the kind of time course that Hull w hinking about when he postu- 
lated that need gives rise to motivation. a : 

Perhaps the most straightforward and convincing demonstration of re- 
inforcement in the absence of drive reduction was reported in several early 
studies by Sheffield and his associates. Sheffield and Roby (1950) found that 
rats would learn to run in an alley if they could drink saccharin in the goal 
box. It was found that it was the vigor with which the consummatory 


response occurred, rather than the nutritional benefit to the animal (sac- 
Juce need), that made saccharin 


and cannot rec 
hat it is not reduction in need or drive or 


Hull that constitutes reinforcement; 
response that produces learn- 


as tl 


charin contains no calories 
reinforcing. Sheffield proposed t 
any variant of this idea discussed by 
it is simply the occurrence of a consummatory 
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ing. The rat learns a response when this res 
explore, or whatever else it is motiv 
summatory response undoubted 


ponse permits it to eat, mate, 
ated to do. The occurrence of the con- 
ly increases the animal's level of — 
momentarily, Sheffield (1966) therefore Suggested that reinforcement may be 
thought of as more like drive induction than drive reduction, 

The evidence against the driver 
to turn up at the time th 
the validity of the 
field's work and was 


eduction hypothesis was just beginning 
at Hull died. But he h 
hypothesis (Hull, 19524), 
evidently aware of othe 


ad already begun to question 
He was impressed with Shef- 

r mounting problems naman 
with his theoretical position. As more negative evidence became ett 
would he have attempted to defend the drive-reduction hypothesis? Woul = 
he have accepted some of reinforcement such as Sheffield 2 
Would he have abandoned the reinforcement principle altogether and tried 
to explain learning without it, as Guthrie and Tolman had done? We do 
not know, but we do know that for several years, when Hull's work was 
being carried forward and defended by a number of colleagues, former 
Students, and independent parties who had been won over to the Hullian 


j k s ; iis f the 
Persuasion, the r “me estion became the all-important issue of t 
day. All during the 1950s the ç 


. r : cement 

¢ Wuestion of what constitutes reinforceme 
and whether Or not reinforcement is necessa 
deal of research 


alternative view 


iry for learning received a gea 
attention. It is not clear why this particular issue shoulc 
have become such a battleground, but it did, Ultimately the battle was lost 
by Hull’s followers, Later in this chapter we shall see that the drive-reduc- 


tion hypothesis has been abandoned by one after another of those who had 
attempted to defend it. 


Inhibitory Factors 


So far we have been talking about excitatory f 
production of behavior. Both D and H contribute 
sion of a response. (But Hull also Postulated the 
inhibit behavior ‘and subtract f 

are two kinds diie 
hibition,” or 7 


actors that lead to the 
Positively to the expres- 


; : y designated “reactive in- 
w IS specific to a Particular p 


al given by the basic 
nto his system to 
> s recovery, The same 
ain the fact that 


sument i 


Opportunity to dissipate 


Inhibitory Factors 103 


next, s z nre a 
, so that the expression of behavior is always somewhat less than if it 


were determined just by the strength of D x H. 
ents kind of inhibition also had a strong Pavlovian flavor. It was 
colhution! 2 itroned inhibition” and was symbolized slp- It was also hy- 
sibl ; © subtract from the strength of behavioral expression. As the 
ndicates, s/p was supposed to be specific to a particular stimulus 
As the basic mechanism was postulated, when a 
and reinforcement does not follow, 
allel to the buildup of sHx 
a. In effect, there are two 
e that makes reinforced responses more probable 
l responses less probable. As we have noted 
1 after extinction is caused by 
l interval. But at the same 
lead to the cessation of 
is caused by the 


and a particular response. 
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g g mechanisms, on 
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uls will ultimately 
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probabilistic. He noted tha 
neth and drive level 


aximize habit strer 
nd distracting stimuli, he still finds a 
He finds to his consternation that 


animals and varies from 
a this variability Hull 


Hull was one of the few theorists to 
ei behavior is essentially 
experimenter has done his best to m 
hs to minimize inhibitory factors a 
ima amount of variability. 

Sponse strength varies widely over a group of 
trial to trial in an individual animal. To deal witl 
Postulated the existence of an oscillation mechanism. He proposed that the 
Overt expression of any behavior requires that the factors, D, H, and J, 
Produce a tendency to respond that is greater than some threshold value. 


A subthreshold response tendency is simply not overtly expressed. Then 
Hull assumed that the threshold of response evocation varies randomly in 
time according to an oscillating function, sÔ r which can sometimes lead to 
the expression of a weak response tendency but 

a strong response 


at other times may inhibit 
the overt expression of tendency. This oscillatory function 
sO, is subtracted from the other deter se strength. The 


Ci > : M 
omplete equation is therefore 


minants of respon 


Behavior strength = DX sHr— In — sIr — 8On 
threshold was not entirely new: it had been 
Ids for many years, but Hull's system 
of such indeterminacy on the be- 


dmission that behavior can be 


The idea of an oscillating 
applied to sensory-detection thresho 
was unique in postulating the existence 
havioral side of the organism. It was an a 
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predicted onl 


y on the average, over 
animals. 


a period of time, or over a group of 

We would seem to have all the conceiy 
gathered before us, but there is still one mo 
to make a particular response can be 
Possible to measure the vigor or 
or force with which it 
probability of respond 
they are not. We migh 
a rare response of gre. 
response? Clearly w 
that will convert t 


able determinants of behavior 
re consideration, The tendency 
measured in different ways. It 1s 
amplitude of a response, that is, the speed 
is executed, It is also possible to measure the rate or 
ing. The two measures may be correlated, but ae 
t have a very frequent response of low amplitude a 
at amplitude. Which are we to say is the “stronger 

hat is needed is some sort of mathematical formulation 
he equation for the strength of behavior directly S 
feet per second, occurrences per minute, or whatever other response measur 

we actually record in the laboratory. Hull tentatively postulated different 
mathematical equations for the different response 
them proved to be entirely satisfactory, 
whose treatment within Hul 


measures, but none of 
But there is one response BURIE 
I's system has proved to be both widely merece" 
and highly convenient. Spence (1954) Proposed, after reviewing a variety 0 
data, that the Probability of a response can be best m 
terms of the reciprocal of its latency. For example, 
as having such strength that it occurs on the average in four seconds, then, 
according to Spence’s analysis, the equation describing the strength of be 
havior, D x H . ; is equal to .25. If the behavioral tendency were twice 
as strong, so that D x H +++ = .50, then the response would occur in two 
seconds on the average. Spence and his students have been quite consistent 
in analyzing their data in terms of such reciprocal time scores, or speed 
scores. This consistency is based partly on the Conviction that such scores 
can be directly related to D xH... and partly on the additional benefit 


that such scores h stical properties. Over a series of trials 
if an animal gives slow ones, the frequency 


a number of fast starts and a few 
distribution may be so badly skewed as to be Virtually useless for statistical 
purposes. Taking distributions much more 


irectiy in 
easured directly 1 
5 ‘ -esponse 
if we think of a respon 


reciprocals typically makes such 
nearly normal. 


Implications of the Theory 


the nervous system. 
ntax. The el ypotheti- 


to the exter situation 


nal stimulus 


Implications of the Theory 105 


ve need not discuss here. And then 


e structure of the theory at two 
on nonreinforced 


by means of additional postulates that v 
this hypothetical stimulus enters into th 
points. It enters into Hp on reinforced trials and into slg 
trials. There will be a tendency to respond when there is an appropriate 
habit—if there is enough drive and if there is not too much inhibition. 
And the tendency to respond is finally transformed according to some 
mathematical function into observable behavior. The value of this type of 
approach was its extreme flexibility; it enabled the Hullian theorist to 
apply the system to a tremendous range of behavioral phenomena. 

All the theoretical relationships that we have discussed here have been 
presented mainly in verbal language. Hull also described them all in tenta- 
tive mathematical equations. Each syntactical link was spelled out formally 


and precisely in tentative equations. It was thus possible to make rather 
and some of the research originating 


athematical in character. 
ing about where various parts 


precise predictions from the theory, 
from Hull's own laboratory was extremely m 

We could indulge in physiological fantasi 
of the learning system are located in the central nervous system. We might 
identify drive as physiologic { locate it in the reticular activating 
System, We might put habit in the cerebral cortex and inhibition in the 
hippocampus. Hull himself was inclined to such speculations from time to 
ume. But they play no part in his system per se. All the constructs are 
Primarily intervening variables (or hypothetical constructs, as Hull preferred 
to call them). The theory is basically descriptive, each part of it being based 
upon certain critical experiments. Hull maintained that the proper use of 
his theory was to predict the outcomes of new experiments and check the 
Outcomes of such experiments against the predictions. Because of the ex- 
Plicitness and relative precision with which Hull's theory was stated, it has 
been possible to test most of his theoretical assertions. Most of those per- 
taining to the drive concept, including the famous drive-reduction hy- 
pothesis of reinforcement, have now been shown to be either wrong or 
Inadequate. Other parts of Hull’s theory, such as his treatment of inhibition, 
have fared somewhat better, xtensively modified by 


but they have been € 
both Hull and his followers: Between 1943 and 1952 Hull made a great 
Many minor changes in the p 


ostulates. For example, reinforcement was no 
longer attributed to reduction in need or dri 


ve, but to the reduction in the 
stimulus correlated with drive. The amount-of-reinforcement variable was 
no longer considered to affect the rate of learning but was made a motiva- 
tional variable (a development that we shall consider in more detail shortly). 
The oscillation function was altered. All these changes came about as a 
result of evidence that accumulated in a few years after Hull's theory was 


first announced in 1943. 
from all the changes that Hull made in his theo- 


It might be inferred i Jull 
retical system that it initial form must have been quite inadequate. But this 


al arousal anc 
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conclusion misses the point of what Hull was trying to do. What een 
to Hull was not whether the details of his postulates were right or scien 
but rather the method for building a theory. Hull no doubt expected al 
the specific details to be modified, corrected, or discarded. The eee w ts 
originally based on very little evidence, but that did not matter. What w as 
important to Hull was that his conjectures could be tested and dtanged or 
rejected as the evidence necessitated. Hull's willingness to be wrong was a 


remarkable, perhaps unique, virtue. It is a virtue that is, unfortunately, not 
shared by many of us. 


The Secondary Learning System 


So far we have considered what may be called the “primary learning 
system.” Hull had, in addition to this D x H system and superimposed 
upon it, what we may call the “secondary learning system,” which adds an- 
other whole dimension of flexibility and power to his theory. Hull was 
aware that many instances of learning found both in everyday life and in 
the laboratory occur in the absence of any apparent reduction in need. He 
therefore assumed that learning can be obtained by means of secondary 
reinforcers. He postulated a class of primary reinforcers, the effectiveness of 
which is not dependent upon prior experience; food for the hungry animal, 
relief from pain, and similar events are typical examples of primary, or 
unlearned, reinforcers. But there is, he assumed, a class of events, like social 
approval and money for the human subject, and getting to a box where food 
has been located for the rat, whose effectiveness as reinforcers depends upon 
prior experience. These events are secondary, or conditioned, reinforcers. 
Hull paid relatively little attention to secondary reinforcement, but he 
included it among the postulates. He also speculated about what is neces- 
sary to establish a stimulus as a secondary reinforcer, The necessary condi- 
tion was said to be pairing of the stimulus with primary ; 

It is interesting to note that secondary reinforcers 
a Pavlovian procedure, Hull accepted Pa 
tioning (using an established CS 

lent to secondary reinforcement. 
in the same way that a prev 
should be noted th 
response but rather 


reinforcement. 
are established through 
ivlov’s claim of higher-order condi- 
as if it were a US), and viewed it as equiva- 
The CS becomes a US, according to Hull, 
iously neutral stimulus becomes 
at what is learned under these 
a new function 
Hull formulated another 
motivation. Hull's primary 

nized a source of learned 

phenomenon is best illustra 


a reinforcer. It 
circumstances is not a 
al property of the stimulus. 


important principle of secondary or learned 
motivator was drive, but in addition he recog- 
motivation, which he called “incentive.” The 
ted by the classic experiment of Crespi (1942). 
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ie a ieee groups of rats to run in an alley for food. Different 
ra - “cme amounts of reinforcement. One group received a 
eal a sie ood ii the goal box, the second group received 16 pellets, 
metis tials eap received 256 pellets, virtually a full day’s ration. After 
sie Dees under these different conditions, the three groups evidenced 

y nt levels of performance m the runway, as shown in the left part 


of Figure 6 $ 
ar igure 6.2. At first, Hull (1943) suggested that such differences in behavior 
ect differences in habit; more food mean: 


vida ung“ iebgse 2” > s more reinforcement, which 
secs at as is c np of habit. But Hull changed his interpretation be- 
eadar 0 r results reported by Crespi. After the twentieth trial Crespi 
Pours Gi ep of reinforcement for all groups at sixteen pellets. As 
shite a : indicates, there were rapid changes 1m peronnancs following the 
sped AAE of reinforcement, all groups quickly adopting a running 
D : t ” as appropriate to the new conditions. Notice that the shifts in 
i ance of the l-pellet and 256-pellet groups were much more rapid than 
et in performance in the original learning. Crespi therefore con- 

that the amount of reinforcement variable does not affect learning 


habi Faas 
(habit) but does affect performance nd of motivational 


wart through some kir 
arig Š o i j A 
iable. Hull (1952a) accepted Cresp! $ argument and designated with the 


postshift 


56-16 
2! fing Pa 
N \ 7 


preshift 


Running speed (feet per second) 
= 1: 


Trials 

ring rats different amounts 

of reward, either 1 OT 16 or 256 pellets. Starting 

with the twentieth trial, all animals were given 16 
Quantitative variation of in- 


pellets. (From Crespi, 
centive and performance in the white rat, American 


Journal of Psychology» 1942, 55-) 


Fig. 6.2 The efject of git 
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letter K a new construct, which he called “incentive motivation.” Hull 
simply inserted K in the behavior equation, so that 


Behavior strength = K x D x Hpo 


Hull now had two kinds of motivators, the old primary drive factor and 
the new secondary incentive factor. He also had, 3 
learning: the primary $-R learning embodied in sHp and the learning in- 
volved in establishing secondary reinforcers and incentive motivators. In 
the first instance it was clearly an S-R connection that was learned, but in 
the second what appeared to be learned was a new property of a previously 
neutral stimulus. Hull made everything consistent, however, by assuming 
that the secondary learning system was based upon the learning of a Ree 
Sponse—a special kind of response, to be sure, but still a response—which 
followed the same laws of learning that he had elaborated in the primary 
system. Hull's argument was that a unique response, such as eating, occurs 
in the goal box. This goal response, designated Rg, is assumed to become 
associated with the stimuli present in the goal box. This argument is very 
much like that noted in our discussion of Guthrie's interpretation of mo- 
tivation. One difference was that Hull invoked drive reduction for the 
learning of all responses, including Rg, but the rest of the argument is the 
same.? Then, if other stimuli in the apparatus are similar to those present 
in the goal box, there will be a tendency for the goal response to generalize 
forward to other parts of the apparatus. Of course, there is no food any- 
where except in the goal box, so the Ry cannot really occur. Certain frac- 
tional parts of Ry can occur, however. Specifically, responses such as saliva- 
tion can occur anywhere in the apparatus. Hull designated this hypothetical 
response a “fractional anticipatory goal reaction.” He expressed it Ya (pro- 
nounced “little ar-gee”). Thus rg is elicited by stimulus generalization by 


stimuli similar to those present in the goal box where Ry occurs, 

There is one more Step to the argument, 
assumed to provide incentive Motivation, w 
symbol K. How can the occurrence of 
answer in Hull's system is that ro 


in effect, two kinds of 


The occurrence of ro was 
hich Hull designated with the 
a response produce motivation? The 
has stimulus consequences. The exact 


; 2 When Hull first developed the Re concept i 


tioning as a universal learning mechanism, But 
reduction reinforcement 


ning, he inv 
the explanation of Re learni i 


sed in a Pavlovian experiment 
xpected to reinforce sa and produce learning 
pter 8, have tended to follow 
a classically conditioned response. 

at conditions Ry, and produces 
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identity i i i 
a a apa re propa e e a 
rence of rg thus introduces a new aide s sat eee - ty? eee oid 
tern. Then this additi i ‘Id. li k vane on mr 
vis add itional stimulus sg could, like any stimulus, have the 
Sa aA cimen w it. The rat thus runs in an alley, partly 
bx Pune ng is associated with alley cues but also partly because alley 
o, which produces Sg, tO which running is also associated. 
Hull's final explanation (1952a) of Crespi's experiment was that animals 
receiving a larger amount of food have a more vigorous Te conditioned to 
the goal box (or perhaps have the same rg conditioned more strongly). In 
oe a will be more strongly elicited by alley stimuli, and there will 
i an oreproiinent Sg to provide additional stimulus support for running. 
was this additional source of stimuli that Hull designated “incentive 
oo or K, and assumed to produce multiplication of D and H. 
1en the amount of reinforcement was suddenly shifted in Crespi’s experi- 
ment, a different rg became rapidly conditioned to the runway stimuli so 
that different amounts of rg and Sq were produced, and a different magni- 


tude of K was generated. 

The development of the re aspect of th 
secondary learning system, may seem unwarranted 
plicated in view of the power and flexibility of the primary 
during the 1950s a variety of facts began to be discovered that simply 
not be dealt with in terms of the primary system alone. Much of the research 


reported earlier by Tolman and his 1 also proved extremely 


students hac 
€ PAREDE. ad a > 
mbarrassing to S-R theory. The re ent if 


mechanism provided a conveni 
not very simple explanation of many of these findings. Consider again the 
hich ra 


ts performed poorly on the first few 
oal box, and then showed a dramatic 


correct respo nse asso 


e theory, and indeed the whole 
and unnecessarily com- 
system. But 
could 


la . i 3 
tent-learning experiment in W 


trials w ~ 
ae us when there was no food in the g 
impr g Nene ‘ 

Mprovement in performance when food was introduced. Tolman had 


poas that learning had been occurring all along but that it became mani- 
est in behavior only when the goal box contained an object for which the 
animal had a demand. Hull's explanation of the same data was that learning 


had indeed been occurring all along (we may note that there was some 
decline in errors by both the control animals and the experimental animals 
before the introduction of reward) but that, although sr had built up, 
cause incentive motivation was so low. When 
an appreciable amount of K 


the goal box, 4 
“multiplied” the previously established 


gaps was very poor be 
the rat encountered food in 
en taal established, and it 
abit, 

For another illustration of the incentive-motivation principle, consider 
the place-learning versus response-learning experiment, in which rats trained 
to run from the south to the east for food continue to run to the east when 
they are tested from the north. Tolman had maintained that such behavior 
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indicates that the animal has learned wher 
the food place. Hull's interpretation w 
the animal looks to the right 


e food is and is simply going to 
as that during training, whenever 


(east), it will encounter stimuli, perhaps extra- 
maze cues like those present in the goal box 


elicit rg and produce sę. Then, when the 
this drive reduction w 
Later, when the 


, and that these stimuli will 
animal runs to the right and eats, 
ill strengthen the running response to these s,; stimuli. 
animal is tested starting from the north, it will again run 
to the choice point and look in both directions; when it looks to the left 
(east), these sọ stimuli are reinstated 
In short, the TES, Mech 
variety of phenomen 
system alone. 

Hull formulated an additional second 
lated that a stimulus th 
secondary source of 


and they tend to control the behavior. 
anism permitted the Hullian theorist to explain a 
a that could not be explained by the primary learning 


ary learning mechanism. He postu- 
at is paired with drive will itself come 
drive. For example, a 
placed in a box where it has been hungry, 
if placed in a box where it has | 


to serve as a 
rat should become hungry if 
It should become frightened 


>een frightened. Hull did not pursue the 
implications of this principle or do any esearch on it, but others did. 
During the 1950s an enormous amount of research was 


of secondary or learned drive. The only instance in which the mechanism 


appeared to work as required by the theory was that of fear conditioning. But 
the results of these experiments were subject to a variety of interpretations 
and provided little support for the learned-d 


e sup rive concept. We shall consider 
this complex subject in more detail in Chapter 9 


addressed to the idea 


Applications of the Theory 


The 1948 version of Hull’s theory was boldl 
tific program and as a set of postulates to en all behavior. ‘Recall 
that Hull’s earlier research had been done with human f } E EN t he 
had brought to his major theory-building efforts a s = byects and tha 
learning. Actually, his first attempt to build “ vackground in pee 
was related to human verbal learning (Hull et al mathematical mode 


conducted during the 1940s to te 1940). But the research 
exclusively with rats, and in 1 


havior system was probably 
resist the temptation to apply a limited s 


Y proclaimed as a new scien- 


nEry rats. But few can 
ystem uni a a > 
TS, Were fee All of our earlier 
* i s, uite ready i 
systems to the explanation of social behavi ady to extend their 


or, abnor ee) 
mental psychology, and the remotest frontiers of eye eens develop- 
appeal and popularity of Hull's theory were no doubt ie tio oleae 
© the program- 
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matic promise of handling problems in these far-flung areas, but its greatest 
a in the area in which its basic postu- 
at in the laboratory. 

a much wider variety of learning 
We have already seen applica- 
ena, but two other 


success was in accounting for the dat 
lates had been derived: the study of the r: 

Hull's theory was designed to handle 
phenomena than any prior theory had been. 
tions of the theory to a number of behavioral phenom 
basic phenomena must be mentioned. One is generalization. Hull treated 
generalization in terms of stimulus similarity, and he recognized that the 
question of similarity is basically a perceptual problem. But he also recog- 
nized, that many perceptual problems can be dealt with quantitatively by 
using psychophysical techniques. In an early experiment Hovland (1937) had 
done psychophysical experiments with human subjects and had found that 
response strength varied in a systematic way (see Figure 6.3) with the dis- 
tance of the test stimulus from the training stimulus. The distance was 
Measured psychologically in terms of just noticeable differences, that is, the 
Number of detection thresholds separating the stimuli. Thus, similarity did 
not have to be judged subjectively; it could be measured along particular 
Physical dimensions, and generalization gradients could be derived from 


ë 


468 1000 1967 
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Frequency 


conditioned galvanic 
rent frequency 


skin 


Fig. 63 Amplitude of 1 ; 
response (GSR) when tested with diffe E 
tones. The arrow indicates the frequency at which 


each group was initially conditioned. The frequen- 
cies were selected to be equally discriminable, but 
are plotted here on a logarithmic scale. (From Hor: 
land, The Journal of General Psychology: 1937, 17, 
125-148, by permission of The Journal Press.) 
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such measurements. In short, 

analysis of stimulus generalizati 
one cared to measure the gen 
assumed to be 


it was possible to provide a T 
on along any stimulus dimension on Da 
eralization gradients. These gradients wer 
an intrinsic property of the sensory receptors. } ae 
Discrimination was easy to deal with in terms of the OPPOSINS Sg ans 
slp constructs (Spence, 1937). The slp builds up to the negative ee 
and generalizes throughout the stimulus dimension: sHp builds up to 
positive stimulus, and there will again be generalized habit strength roe 
other stimuli along the stimulus dimension. The resulting strength of 


t x Ta P the 
havior is assumed to be the difference between the positive Hp and 
negative Ip components. The scheme is illustrated in Figure 6.4. 


Response tendencies 


Stimulus dimension 


Fig. 6.4 Illustration of Spence’s (1937) theory of 
generalization and discrimination, The higher, heavy 
line represents the tendency for the excitatory effects 
of training with S+ to generalize to other points on 
the stimulus dimension. The lower, lighter line rep- 
resents the generalization of inhibition resulting 
from training with S—. The broken line, which is 
the difference between t 


he two, represent. 
sulting tendency 


S the re. 
of the response to occur to 
stimuli. 


different 


Further Developments 


During the critical 
together and modifyin, 
of being the center o 


years of the 1980s and 19405, When Hull was putting 
g his behavior System, he had the į i 
f an extremely active and ¢ a a see 
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able men to carry forth the program he had started. These men tested his 
ideas, as well as their own, and continued to modify the basic postulates and 
refine the whole theoretical structure. Many of these men had able students 
of their own who provided a third generation of Hullian or neo-Hullian 


theorists, Thus the enterprise that Hull began at Yale spread around the 
chologists who were not dedicated to 


country in just a few years. Even psy 
aught up in such issues as the nature 


Hullian theory itself found themselves c 


of reinforcement and the nature of incentive motivation. 
alysis of all the work or the issues involved, but 


we may survey some of the highlights. Some of the basic assumptions of 
Hull's 1948 theory had been anticipated by his colleagues. The importance 
of motivation as a codeterminant of behavior (along with habit) had been 
Stressed by O. Hobart Mowrer. In two important theoretical papers Mowrer 
(1938, 1939) showed the inadequacy of a purely Pavlovian approach to 
learning, and he began to build a specialized theory of avoidance learning, 
an extremely challenging phenomenon that no one but the Hullians, and 
Particularly Mowrer, was able to deal with at all adequately. In these same 
papers Mowrer had also urged that learning by reinforcement be substituted 
for learning by contiguity, which was still in vogue at that time. Mowrer 
began to part company with Hull in 1947 and continued to go his own 
way. He soon began to emphasize the importance of the secondary learning 
system, and in his 1960 book there was almost no vestige of Hull's original 


primary learning system. i 7 
In 1941 Neal Miller and John Dollard presented a systematic and com- 


prehensive behavior theory that anticipated many features of Hull’s 1943 
as not as formal or as detailed and complex. The 


system. The presentation wa s : $ 
habit construct was essentially the same. The drive-reduction hypothesis of 
reinforcement was there, but drive was treated considerably differently, not 
as a separate kind of construct. According to Miller and Dollard, any strong 

s without being tied to the 


stimulus can have motivating Or drive properties WIN” Poke 
needs of the organism. Complications such as inhibition and oscillation 


were missing from their theory. Extinction, for example, was explained by 
means of competing responses rather than by inhibition. This simplified 
version of, or preview of, Hull’s theory was shown to RANE great potential 
applicability to human social learning. In subsequent years Miller came to 
dominate both theory and research in the area of conflict (for example, 
Miller, 1959). He, together with Mowrer, clarified and attempted to codify 
the difficult areas of learned drives and learned rewards (Miller, 1951). 
These developments are so important for understanding the subsequent his- 
tory of learning theory that we will devote more space to them in Chapter 8. 
In the 1950s Miller turned his attention tO the relationship between drive 
and need and the relationship between drive reduction and reinforcement. 
Much of his work on these problems is summarized in an important paper 


Space does not permit an 
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i : pothesis 
(Miller, 1957), in which he concludes that the drive-reduction hypo 


«odd it to 
can be defended, provided that drive is not tied as closely as Hull pen 
biological needs. In recent years Miller has focused his avenuon 
physiological bases of learning and motivation (Miller, 1969). lefined 

A few theorists have attempted to follow the principles uyat ended 
Hull's primary learning system. For example, Judson Brown (1961) de -oi 
the concept of drive, and with considerable success, but his definiti 
drive was somewhat different from Hull’s orig 
expanded to include learned sources of motiv 
recent years most Hullian theorists h 
the primary learning sy. 

The man who was 
Secondary system w 
Spence had contrib 


inal definition, in that it pi 
ation, such as incentive. But : 
ave followed Mowrer in abandoning 
stem in favor of the secondary learning peor ‘ita 
mainly responsible for the early development in ba 
as Kenneth Spence. In some important early ane 
ited to the Hullian enterprise in such ways as ana y fives 
discrimination learning (Spence, 1937) and clarifying the logic of the we li- 
matical deductive approach (Spence, 1944). But it was Spence wylio S oe 
cated the rg theory of incentive motivation (Spence, 1951) and stresses sae 
importance of secondary reinforcement (Spence, 1947). His last majo} i irit 
retical statement (Spence, 1956) retains so much of the programmatic Tiat 
and scientific philosophy that had characterized the Hullian enterprise e 
it constitutes, in effect, a final Status report on Hull’s theory. But at a 
same time Spence shifted the emphasis so much and broke so much ne 
ground that his book stands apart from Hullian theor 
the end of the era. For example, Spence gave a gre 
Such phenomena as amount of reinforcement 
Much of the relevant research h 
explanation seemed to require 
D x H equation w 


y and perhaps mapa 
at deal of attention t 
and delay of reinforcement- 
by his own students, and i 
the secondary learning system. The old 
in describing these phenomena. 


During the 1950s his ideas completely dom! 
ature. Thus he accomplished what he wanted to do, 


research that would test his theoretical conjectures: 
Virtually every behavioral conjecture that Hull made has now been shown 


to be wrong. But that is not important; what is important is that, because 
of Hull, we now know a great deal more about the hungry rat than we did 
a a 


ad been done 


as of no use 

Hull’s impact was immense, 
nated the research liter 
which was to generate 


just a few years ago. 
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é vior § 1952a 
are tough going and require extraordinary commitment from .. f o 
shorter works are more rewarding but typically Present on PUN 


ly part of the total theo- 
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ne system. Spence (1956) is the best spokesman for both his own and Hull's 
(1959) pis many theoretical issues; there is also an excellent analysis by Logan 
Spence’ er common philosophy of science is further described in several of 
a sey x collected papers (Spence, 1960). Hull’s motivation principles are discussed 
first me detail by Brown (1961). For biographical material we are fortunate to have, 

» an autobiographical chapter (Hull, 1952b) and, second, the “idea books,” a 


Sort of personal intellectual diary (Hull, 1962). 


r studying behavior, ; E. Skinner (born 1904) has 


alyzing it, ana o Wivapparatus for observ- 
wg 1t, and his own idea: 


atid i s about how it 
à AEN a . 

Skinner’s methods and his approach to the picky. an accelerated pace, 

become popular. Their Success is apparent not on] cms of behavior have 

but also in a broad range E 


ications, g, -P Oatory situations 
S. Over th i $ 
e year. ner 
owers who, if years Skin: 
to call the 


not willing to call 

z A r s 

tioners and who vigorously extol the virtues of ae Operant condi- 
M . 1 ` 

analysis of behavior. nner’s €xperimental 


116 


Skinner 117 


Our i 
earlier theori 
orists r i é A 
sts had all struggled with the basic term “stimulus.” 
e object in the environment; 


For T 

it Paea stimulus was not a manipulabl 

therdesutnance E hypothetical events whose existence was inferred from 

only hough a Sexi hebasten Hull tied behavior to stimulus objects but 

poatilertes: nandi uous path of intervening variables and a number of 

almost totally : g one postulate that could make the effective stimulus 
ally unrelated to physical stimulus objects. For Tolman it was 


not the sti 
stim i A s * S 
In his first ae object itself but how the animal perceived it that mattered. 
st theoretical paper, which had been part of his doctoral disserta- 


tion Ski 
» Skinner (19 od 
come to er (1931) observed that the terms stimulus and response had both 
acquire a 3 5 z 
andres quire a double reference. Even the relationship between stimulus 
ponse, the reflex, had a double meaning. On the one hand, there was 


the | 

lypo : . z š AÑ x 

ypothetical physiological entity, the “reflex arc, which was assumed 
iven receptor was stimulated. On 


to ; 
T 3 given movement when a gi or ; 
by Retin at there was the fact that we can often elicit a reliable response 
fuse the ee an appropriate stimulus object. Difficulties arise when we con- 
Consider mR S-R with the assumed underlying physiological S-R. 
absence of = that when Pavlov first discovered his dogs salivating in the 
any physiological event that could cause the secretion, he found 


1 
t necessary to coin the anachronistic phrase “psychic reflex.” Here the be- 
but the underlying mode of operation was 
as like a physiological reflex. 
logical s t es rise to both the psycho- 
hee concept of reflex and the physiological concept of the reflex arc. The 
with e is that certain specific responses are very reliably correlated 
ertain specific stimuli. The empirical or behavioral reflex is, then, 
tion of a response with a stimulus. The 
tis partly speculative and partly the 
t have been discovered by the 
basic datum for the student 


f stimulus and response. 


havi 
viorz 
ANS events were clear, 
ur í 
Skin re, and Pavlov merely assumed that it W 
er tells us that a single basic observation g!V 


Poles: mR than the correla J 
result age machinery that produces i 
Physiol further empirical correlations tha 
ogists. Skinner (1931) concluded that the 


of oE i 
behavior is simply an observed correlation O 
it Robert Whytt discovered the pupillary 


liscovered either the contraction of the 
he retina, put rather that he first 
these two events. So far as be- 
g more than this rela- 


When we say, for example, th 
reflex, we do not mean that he ¢ 
iris or the impingement of light upon t 
stated the necessary relationship between 
havior is concerned, the pupillary reflex is nothin A 
tionship. Once given a specific stimulus-response correlation, we may, of 
course investigate the physiological facts of its mediation. The informa- 
tion there revealed will suppleme ition but it will not affect 


the status of the reflex as a correlation. (Skinner, 1931, p. 439) 


ely high, and we.are all 
when it is high enough. 
sed many years before, that 


the observed correlation is extrem! 
word “reflex” 


lov had propo 


Sometimes 
pretty well agreed on using the 
Skinner proposed, however, as Pav. 
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eak 

3 Telat even a w 

we use the word “reflex” kind of Bi a a quite 

one, between stimulus and response.! Thus we see Skinner se 3 meandi 
F + A x fi n 

early that for him the explanation of behavior is to be foune 


zor should 
i i s APOR science of behavior sh 
tions with which the behavior is correlated. The science of be 
concern itself not with either 


lying mechanisms | 
make our obsery 

In a further a 
the problem of 


to describe a ny 


x r under- 
postulating or attempting to Te A will 
DUL with devising experimental arrangements tha 
ations more lawful and orderly, 354) addressed 
nalysis of the concept of stimulus, Skinner (19352) eae 
the relationship between stimulus objects and Hie Sera 
nants of the animal's behavior. His answer was again essentially “ae to 
We have to recognize that the physical energies change from one ails 
the next and that, as Guthrie observed, what we like to call the is, in 
is in Teality a vast Population of potential stimulus elements. ee cor- 
short, an indefinitely large class of events that the response may “ically: 
related with, But how large this class is can only be determined empirie 
If we vary the stimulus situation we will 
elements is relevant, in the sense th 
response, whereas another 
modified without 
to the response si 
be effective in de 
in a Skinner box 
probability when 
response for us. 
events produce a given corre] 
defined as whatever motor 
stimulus, Later, whe 
it proved more convenient to define 
sequence, so that in the Skinner 
whatever motor Movements 
pellet. Here we find 


always find that some subali 
at it maintains the correlation nah be 
subclass is irreley 
altering the corre] 
de. There is a l 
pressing a lev 
» but there is 
the stimulus 
In effect, then, 


ant, in the sense that it om 
ation. The same argument can be il 
arge class of muscle movements that a 
er sufficiently to operate the food maga 


res in 
changes 1 
only one rele “ant subclass that chang 
is changed, 


and this subclass defines the 
the stimulus 
ation with 
activities m 
n Skinner (1938) a 


is whatever environmental 
response, and the TORI e 
ain a given correlation wath a 
€veloped the concept of the opera ee 

erms of some fixed a 
box, > à response was defined a 
7 produce ssion of the lever and a foot 
d a different correlati the idea 
So, from the beginning, Skinner di 
cally defined unit of analysis, The 
reinforcement—al] Of the terms th 


a 
aint 


is the same. . 
i 3 nl- 
ae himself to any anaton 


E e 
the response, and later th 
at enter into Skinner's analysis of be- 
1 Pavlov called all behaviors ecause 
that they be thought of as i 


call all behaviors reflexes because 


S 
Skinner (1931) wanted to 
i SI JS 
, to wa ir must have seemed the best way to indicie 
that they are simply S-R corre ations. For Pavloy the Cchanistic Mie et antie 
only way to sati fy his requirement that the stimulus n essarily relation was 
; ity i i y pr i 
whereas for Skinner necessity itself is only a Matter 4 Produce 
require a mechanical system; it 


events be perfect or nearly perfect. 
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havi 
avior—wer 
e defined i oy 
ned in terms not of an @ priori conception of what these 


units n 
nust be b i 
u , “i 
observed Ste simply of classes of observable events that maintain the 
ations between them. If we find that we can control a minute 
for example 


movement like $ 
Bae ia wel of an individual muscle fiber (see 
inent RS Harford, 1958), then for that purpose that move- 
oing to the movi okope If we find that we can control a molar act like 
appropriately defi S c contingent reinforcement, then that behavior is 
; By the Ts a oe repon 
ditioning pas . 30s something of a crisi 
universal ee ee longer be viewed as either the universal method or the 
Geconte siras ss for producing learning, yet the term conditioning had 
to be called 3 3 ponymonus with learning. A learned response was likely 
bysheang. of f oa eine response” whether the learning was obtained 
Settle the ie aypan or a ThersGhe procedure. In an attempt tO 
atter equitably, Gkinner (1935b) observed that there are two 


types of 

s of cna p 
conditioned reflexes—tWo0 types of learned S-R correlations—and 

erent experimental procedures. Then, partly 


and partly because of the different kinds 
arning with them, there are certain other 
important event S* must necessarily 
forcement procedure; but S* pre- 
a Pavlovian procedure. A second 
ned reflex may have zero 
ur at least 


s had arisen because Pavlovian con- 


tha Be 
eee produced by diff 
of fechas re different procedure 
technical En oe subject to.le 
follow os es For example, the 
cedes the ES response 1N the rein 
point of in and usually elicits 1t in av 
strength difference is that the cl conditione 
gth initially, but the operant ¢ ise it has to occ 
forced. 
mulus that he assumed must 


mentions the sti 
a the operant response. He said that it was 
is diagrams. But Skinner must have 


hypothetical, ast unobserved, 
al analysis. The elicitor of the 


assically 
annot becat 


E before it can be rein 
enka 1935 paper Skinner 

sarily be correlated witl 
ided it in h 
a 


uni p 
fe mportant, but he inclu 
a the inconsistency of including 4 or at le 


Stir 2 Ż A . par 
A nulus in his otherwise highly empiric 
perant began to disappear in 1937 and was totally absent from his 1938 


book. A new and much more fundamental difference between the two learn- 
Ing procedures was discovered. The classically conditioned response is cor- 
related with and controlled by an antecedent event, an eliciting stimulus, 
which is initially the US and subsequently the CS. But in the reinforcement- 


learning paradigm there is nO antecedents behavior is controlled by its 
Although there may actually be an internal, unobservable 


f the behavioral scientist, We recall, is not to speculate 
about or search for such things- His task is to discover what environmental 
events control behavior- In the case of the operant response it is correlated 


with its consequence "The operant 1s emitted, rather than elicited, and it is 
controlled by its con: In the Pavlovian procedure the response 


consequence, s*. 
elicitor, the task O. 


sequences- 
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z je eh z : -its eliciting 
now called a ‘respondent,” is elicited and is controlled by its 


P ya: 2 o proce- 
then became the fundamenta] difference in the two } 


e that different kinds : 
to be peculiarly subject to learning with these ERD, ‘ays 
dures. The Operant is typically a skeletal response, and in earlier d 

ary. Such behavio 


is readily observable because ngs 
» that behavior with which the organism — 
is why it is called operant), and because it is exte ifie 
it is Behavior that the environment can control by reinforcement and rai 
ishment, Respondent behavior js internal and has a secret, or eee 
quality about it. Feelings, glandular secretions, or emotional TEAGEN dọ 
mmunicated to the environment/and our A 
to reinforce or punish these kinds of reactions.” al 
wo classes of behavior: The respondent clas aater “ 
of emotional and glandular reactions (like ne niet 
1S more public and consists of skeletal Deliaviors, vine 
i l frequently meet with particular ore 
ass seems peculiarly subject to classical OF d 
erant class seems peculiarly subject to modification an 


3 3 k o kinds 
control by environmental consequences, Two kinds of behavior: two ki 
of learning, y” 


and consists primarily 


rapidly developing his GIE TE sve are to believe his 
Own account (Skinner, 1956), the Ski r 


rderly and lawful. 
a small enclosure $O 
a lot of running 
» SO that the rat 
ssing a small mechanica] levers ie aging defined 
8 to Judge if the response had occurred, 
ecause a, a “oie, Indeed, Skinner 

a Mechanica] response- 


that their time, and Skinner’ 


s time, was not 
around, A response th 


o 
at required little ef 


2? We will see in Chapter 8 that, 


if someone takes 
reactions and bothers to arrange coi 


the trout, 
ble . i 
nsequences for them, they ak Monitor these private 
punishable and can be made voluntary. 


both Teinforceable and 
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recordi 

ng system. Ski 

was also developed eas (1956) has indicated that the cumulative record 

delivery Pisin oad hipaa as a matter of expediency. The rotating food- 
s 1e was using at the time had a central spindle, and all 


that 
was neces 
essary was A 
y was to put a string around the spindle and allow it to 


Wind u 
p as the fi A 
ood magazine rotated. A pen attached to the string pressed 


again 
st a rotati 
ating paper dr 
g paper drum so that the results of an experiment were 
e number of responses made after a 


cumulative record is simply one 
response in time may be 
perhaps more easily than 


Immedia z 

given im dawni as the cumulativi 

of many aiei viie (see Figure 7.1). The 

recorded, bes Sa Wliich the occurrence of the r 

with any other Ea the experimenter to see 
presentation the rate at which the response occurs. 


Total responses 


| oe ma 


Time 
of a cum 


ber of responses ino 
t a higher rate. 
a small 


A small sample nulative record 
the total num reases 
at first and then @ 
rd is constructed on such 
ponses cannot be seen; there- 


rall rate is apparent. 


Fig. 7.1 
showing how 
with time, slowly 


Typically the reco 
scale that individual res 
fore only the resulting ove: 


as born. The methods of obtaining 


So the basic Skinnerian exp? 

data and analyzing it were introduced in Skinner’s doctoral dissertation and 

have been used with little further modification in the bulk of operant-con- 

ditioning experiments from that time to this. What was at first a convenience 
lly a ritual, Skinner and his fol- 


= a while became routine and eventua 
owers rarely 100 except as it emerges in their cumulative 


k at behavior 
records. They rarely look at any aspect or dimension of behavior other than 
the rate of respond easures of response strength such as ampli 


qling- Other M 
tude, latency following the onset of a cue, or the percentage of time 
response is made to one stimulus rather than another are usually igr Fa 
a nored. 


riment W 
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There is another feature of the ritual. Skinner h ee 
that to be useful a law of behavior must be applicable s So aloo 
subject. Empirical generalizations that apply only to the ma aye sino 
ance of a number of subjects are likely to have little power or Be Š ot 
The experimental strategy is therefore to establish and demonstrate ‘“ aed 
ticular kind of control in a single subject. Typically, a stable base-line ri 
of responding is first established. Then some t ar i 
systematically altered, and the resulting change from the original base- 
rate of responding is carefully noted. Then tl 
conditions under which the original 
hoped, the behavior retu 
departure from the base l 


as frequently argued 


experimental parameter 1 


e experimenter returns to the 


base line was obtained, and, it 
rns to the initial rate. 


This proc 
ine and return to it is 


S of systematic 


repeated as often as neces- 
i i j Säi exe? vee the be 
sary to convince the experimenter that he has gained control over th 


havior, that he can manipulate it in the same way that he manipulates, say, 
an electric light with a switch. Then, to show that there is nothing excep 
tional about that subject, the same phenomenon is obtained in one or two 
others. Once the experimenter has found how to control a particular sub- 


ject’s behavior, the technique can be verified with 
lawfulness itself is said to resid 


lawfulness that must be soug 


other individuals, but 
ce in the individual's behavior; it is that 
ht in an experimental analysis of behavior. 
Does this mean that we have finally arrived at a truly deterministic psy- 
chology? Can we leave behind the uncertain semantics of Guthrie, the loose 
syntax of Tolman, and the oscillatory function of Hull? 
ing to many Skinnerians, Behavior is determined in 

lawful, they say, and if we find variability in beh 
not yet learned how to control it fully, 
ment is largely lost, however, 
tions that 


Yes we can, accord- 
the sense that it is 
avior it is because we have 
The force of this optimistic argu- 
because it is based on two special considera- 


are often not recognized. One consideration is that behavio 
appears to become lawful when it is 


analyzed in terms of rate of responding: 

it is the number of r at is highly predictable, But ss nie} 
asure itself based upon a kind of averaging? An animal might 
an hour and stil] show a great deal 

ability. When examined in detail, 


animal pausing after one reinforcement for ten seconds 
before resuming bar Pressing, then Pausing for thirty seconds after the next 
reinforcement, There ing particularly lawful or predictable 
about these pauses, lictability is obscured by looking only 
at the overall cumu In fairness to Skinnerian researcher it 
should be emph 'Ve extensively Studied local rates, short- 
f their 


of moment by moment vari 


a cumulative 
record may show an 


may be noth 
and this unpred 
lative record, 
asized that they ha 
term effects, and the fine | 


details o 


cumulative records generally. But 
it should also be noted that these details of behavior often prove peculiarly 
unpredictable. The fact remains that the mom 


i ; : ent by moment behavior of 
an animal in any sit 


unpredictable and this diffi- 


uation is very likely to be 
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culty ca i 
an be easily E izi 
y obscured by summarizing a great number of behavioral 


ars in a cumulative record. 
Mae consideration upon w 
ike ies ipo that it really 
the eae tet = w produce stability, 
iei Bia wet ea of asympto 
kindi ap i ee thë kind of behavioral control he is looking for or the 
ie seeks until his animal has been run for 50 or 100 


sessions i 
Sr ki m a 7 i i 

1 the Skinner box. During this long time the behavior may reveal 
great potential interest for what 


hich the argument of lawfulness often 
does not matter how long it takes 
to gain contro] over behavior; it is 
tic behavior that matters. A Skin- 


they i pete some systematic and of 

) eal about learning and others that appear to be unsystematic 
But with sufficient training all the changes in 
he more subtle factors interfering 


and the behavior becomes predict- 


anc = 
ia more or less random. 
av t C3 n . 
ith ie caused by learning, as well as t 
e ieg aa 
ective control, become stabilized 

able, á 

an be summarized 


The dev x 

wiit dey elopment of Skinner's approach to behavior € 

1 Š : rea 

e following points. On some of the toughest theoretical issues of the 


mi g 
a oe aia a firm stand on the side of an objective, empirical 
us that er 3 S approach has been both sophisticated and practical; he tells 
cult mena rould not confuse philosophy with data collection. On the diffi- 
again to-la n of how to define the stimulus and the response, he tells us 
sistently si ne data be our guide. If there is a class of events that is con- 
there js i — with behavior th 1 e stimulus, and if 
the aca es ass of events produced by the animal that can be correlated with 
to be ei situation then we may call it the response. There seem 
Which lear inds of learning because o kinds of procedures with 
arning can be demonstrated: e two ways to make 


en we may call it th 


there are tw 
behav that is, there ar 
avian predi 
lor predictable. 
a great deal of 


Ski di i 
nner discovered quite early e to obtain 


a response, bar pressing, 
fined the animal 
press the 


that it is possibl 
, methods. He selected 
1 frequently, and he con 
ttle else to do except 
ays in which it was 
ontrol is achieved 
but this difference 
al subjects can 


Sete ct 
Y ey pi data using certair 
in e animal can make easily ang 

n which there was li 
watch all the different W 
ng. Sometimes C 
leal of training. 
avior of individu 


a si ee r 
bar “Lag apparatus ir 
possil hen he proceeded to 
SSI ` 
ati to control the rate 
is rel X and sometimes it requires a 
* atively ini rtant é a i 
ultim y unimportant as long as the bel 


of respondir 
great a 


ately be controlled. 


The Concept of Reinforcement 

is progr a consequence of a 
f responding is ci 
assumptions 


ammed as 
alled a reinforcer. This 


Any envi 
y environmental event that 
about the under- 


resp 

Spons , 

deface that can increase the rate © 
i $ : Br < 
uon is strictly empirical; 1t makes no 
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lying Process, Skinner h 


shat is 
ion of wha 
as stayed relatively free of the question 

learned when lear G 


re for 
Zua rocedure 
i fe) i rei À is a proc 
ning occurs, r Skinner, reinforcement i 


c- 
conne! 
. - A z res S-R 
controlling behavior, not a hypothetical] device that produc 

tions, habits, or expectancies, 


: the 
s that increase 
Reinforcers are simply events that it 
rate of responding. 


5 f other 
forts to control the behavior heron 
vents are rather consistent i onl 
TY Tat, dog, or pigeon, saod i » across 
as reinforcers like food have great peneman loncing 
fic. Petting a dog can be jim of the 
a rat is usually not. In the won ore 
» praise, or getting the right ini tear UP 

a gerbil pieces of paper that it can 


War 
- earlié 
H 5 2 ur e4 
Particular behaviors (recall o 

d elicits e 
ay elicit shred 


; be- 
i slicited 
ding. It is clear that such elic 

he execution 


; than 
time cating food pelam to 

acteristically practical anini 
inforcement is not given after each response in a cor iter- 
inforcement, or CRF); it is given meee 
make a large number of rapo vill 
of reinforcements. The cumulative record V at 
Pent pressi " rather than time spe! 


: those whose nee 
T the hungry rat, and those pan 
ic shock, The ge 
Ontingency between be 
then we have an effec- 


Procedure, Punishment, in 
is west such as ck is ma, € contingent upon 
ka Weakened, then We have an effective punish- 
re tendency wj e all things as good or bad 
: Negative reinforcers as 
g equally ba > but they are defined 
Tily the Same king. OF “bad” stimuli 
Strength ehavior or as 
interestin ©Xceptions (see 


UP such a c 


punishers to weaken it, 
Bolles & Seelbach, 1964). 
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There isa 3 
A common st kj e category: the removal of a positive reinforcer. 
animal ney hé EA in contemporary research is the time out (TO). An 
second Teose i ng one response to obtain food; whenever it makes a 
no longer sa a Grist TO is introduced during which the first response 
tinea on ices food. Under these conditions it is usually found that 
1e second response is reduced. In these cases the removal of the 


Positive reinfor 

(Some ec reduces the rate of responding. 

are nivel Te ie are effective without prior experience. Food and shock 
primary cation typical examples. These events are therefore called 
is possible to eit — as Skinner discovered quite early in his career, it 
as a reinforcer po ish a completely neutral cue such as a light or a buzzer 
invdduced this hat is, we may have a conditioned reinforcer; Skinner 
gated it Ripert concept several years before Hull did, and he also investi- 
A imentally considerably earlier. Before describing conditioned 


reinf 
orcem me, z x . : 
ent, it is necessary to make an important digression to describe 
a response occur under 


discri Wn 
Binder eerie. everyday life we require that a re 
ment for A aS and not willy-nilly. As the administrators of reinforce- 
and withhold : organisms, we reinforce their behavior on some occasions 
reinforced is a on others. A stimulus in the presence of which behavior is 
fore define s called a discriminative stimulus, S7: The presence of S? there- 
Mentary = when the reinforcement contingency is in effect. The comple- 
y stimulus in the presence of which the particular behavior is not 
ining is carried 


rein, 

orce is . . . . 
reed is usually designated S°. When discrimination tral 
ater is that the animal comes to respond almost 


S? and withholds its behavior in the presence 
analysis, Skinner does not speculate about 
r to be concerned with 


appe# 
ate of responding in Sô. It is 


aining is frequently, though 
l in the presence of one 


Out, wha 
ire happens sooner or ] 
of Sa, D yin the presence of j 
the steeds other aspects of his 

ying mechanisms. He does not 


Whethe 
her some form of inhibition reduces the r 


act that discrimination tr: 
als respond 


e in making anim. 

presence of another. 
is successful, we can say that we have 
trol or under discriminative con- 
because operant behavior is 


or usually care, what 


Si 
sien an empirical fa 
nue effectiv 
W us and not respond in the 
Šon oe discrimination training i 
trol pnt the behavior under stimulus con” 
- There is an interesting aradox here, 
the fact that we do not know, i 
he same time most operant behavior is under some 
and if this control is strong enough we have 
enting the appropriate S?. But 
“elicit” in this context. Primary or un- 
: » whereas learned stimulus control 


P! 


irg Moston of eliciting 
lear not proper to use t° | 
„Sarned control of behavior 15 

18 typically called just “stimulus control 
but it emphasizes an important distinction. o 

ame experiment originally reported by Skinner 


Consider now the 


126 Skinner 


in 1936. Rats were trained to press a bar for food, and then this beliavagt 
was brought under discriminative control in which S? and Sè were pre- 
sented during alternate three-minute periods 
behavior became highly predict 
presence of SP and virtu 
point, Skinner changed t 


- After a number of sessions the 
able, with a high rate of responding in ine 
ally no responding in the presence of Sê. At ee: 
he experimental situation so that the duration P) 
S* was not fixed. The rat could terminate S3 and begin the S”? period bu 
pressing the bar. Soon bar Pressing began to occur in the $9. It is clear ai 
making the onset of Sp contingent upon bar pressing increased the rate ne 
bar pressing, By definition, therefore, the onset of S? is a reinforcer, bu 
because it was not originally a reinforcer it must be 
(Skinnerians say “conditioned” reinforcers; 
inforcers). At this point, Skinner made 
conjectures: SPs are conditioned reinfo 

Conditioned reinforcement is readily 
using the conventional method of initi 
First, the rat is put in the box 
livers a free food pellet from 
accompanied by a ch 
while 


a conditioned reinforcer 
Hullians say “secondary” TC 
one of his relatively rare theoretical 
reers. L 
demonstrated in the Skinner box 
ally training rats to press the bar. 
» and the magazine that holds the food aes 
time to time, The presentation of food 1s 
aracteristic “click” from the food magazine. After 4 
this click becomes an sp for approaching the food cup (because food 


reinforces approach following the S” but not other times). The click can 
be established as a very effective SP, 


so that no matter what the animal 15 
doing or where it is located it will run Over to the food cup when the click 
occurs. Now the bar is introduced for the first time, 
can sit back and wait for 


the animal to make 
does, the click occurs, and the rat ru 


a new phenomenon appears. 
The argument (there is ve ry i vide 
argument) is that, just z i Pproach response, so the click 
reinforces the bar-press Provides additional delayed 
primary reinforcement for Pressing, but the initial contact with the 
behavior is made by the immed 


A r 
and the experimente 
x t 
its first response. When ! 
its food as before. But now 


t ultimately 
called bar pressing consists of a series o 


» Or chain, 
must approach the bar, 
the food, eat it 


integrated into the unit 
of responses, The animal 
BO over to the food cup, obtain 


nse in the chain is said 
of food but 


Ponse in the chain. 
executed efficiently, 
reinforcers tightens up 
Produces what 


avior occurs it is not 
les of Secondary 


Sequence, and I 


but the repeated action of the ser 
the behavior, speeds up the whole 


amounts 
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to a single uni x 

unit can TAS behavior that occurs rapidly enough so that the whole 

the summary ioe aes immediately reinforced by fhe: food. Again we have 

One way to com cucal statement that SP is a conditioned reinforcer. 

criminative cant ig the specificity of behavior is to bring it under dis- 

We may Wish t rol. But there is also another kind of response specificity. 

force, ae rie H have a rat press a bar with between 15 and 20 grams of 
y less and not any more. We may wish a child to say “daddy” 


Mstead of wada OF wi 
ada, his cr . . . . 
This can be done by a technique of differential reinforce- 

ng. 


ment 

Eat eA 

ý at is similar . stot n Fa i$ 

ticular stimuli yilar to discrimination traini Rather than specify par- 
i under which the contingency is in 

1 particular subcl 


effect, we require a 
ass of the organism's total 
nent to just those responses that we 


asonable operant level, so that 
n in reinforcement 
as the 


Particular 
ee form of response, i 
want, If a We restrict reinforcer 
the total on appropriate subclass has some re 
density fo ees of behavior will survive the reductio. 
T NER extinguishing, it should gradually become stronger 
the desired erat Sei goes into effect. If the original operant rate of 
Mtroduce a n payor is too low to survive the new requirement, then we can 
rat is pine te trick: We can “shape” the behavior. For example, if the 
May RAR the bar with a tones that ranges only from 5 to 10 grams, we 
is, from 7.5 e responses that lie in the upper half of this distribution, that 
increase on tg 10 grams. As we do so, the mean response strength will 
ing a St if the variance stays the same, the animal will:scon be show- 
inforce ka ution of forces ranging between 7.5 and 12.5 pami 1 we re- 

nly the stronger half of these responses, the distributions should 


SOon shif l 
givi shift to a range between 10 and 15 grams. Soon the animal should be 
Aan, 5 2 i è 

& us precisely the behavior we want. 


Schedules 


duces further specificity into 


igen is another mechanism that introd i Á 
behavior, Skinner noted that reinforcement in the real world is typically 
a invariant. So, if we are to understand behavior in 

dules of intermittence (we have 


aent rather thar 
: real world, we must study 
already noted that intermitten 
erating lots of behavior). Differer 


o E: : i 
wn characteristic alterations 17 
press, W 


different sche 


į schedules serve the experimenter by gen- 


nt schedules of reinforcement produce their 
the rate of responding. For example, if we 
eae emer tl ae e typically find the rat giving us five quick 

rief pause while it consumes its food pellet, as in 


responses and then a b l 
Figure 7.2. If we reinforce every twentieth response, we are ikek to finda 
but we are also likely to find 


very high rate of responding during a run, 
longer pauses following reinforcement. If we train an animal up to a ratio 
a ra 
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of responding on different FR Schedules. The little 

hash marks indicate delivery of reinforcement. 

a auses 

of 100, we may expect to find breaks in the run and long, variable p 
after reinforcem, i 


: number of responses (a VR schedule), 
high fixed ratios tend to disappear A VR-100 schedule 
Postreinforcement Pause. p ield long steady 
runs. i 


f 
only after a fixed lapse ie 
© first response that occur 
: N such a 
uced, we tend to obtain €xting 


nex 
Scallop.” 
acquires discriminatiy, 


reinforcement comes due. 


'S as if the lapse of time 
€ contro] Over responding- 
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Short postrei 
sities oe SSIS times constitute an Sê, and times a hi 
dictable. f ome an SP, Again, if the time for reinfor e P K 
scallops d or example, if we have a variable PER Pe 
schedule © not develop, and the animal oe us. F oa R 
ule, a AEEA Deg al g , as in the case of the VR 
Consider arai y, continuous rate of responding. 
secondary.reinfore the schedule of reinforcement that Skinner used in the 
bar fon food EE study already cited. His animals were pressing a 
1n extinction fo 5 period of time in the presence of one stimulus and were 
a program is ath period of time in the presence of a second stimulus. Such 
m or more pa ed a “multiple schedule.” A similar schedule that includes 
happening at ; mponents but without correlated stimuli to indicate what is 
also be subj a given moment is called a “mixed schedule.” An animal may 
called ah oi to two reinforcement schedules at the same time, a SO- 
urrent schedule in which two different schedules, two different 


kir 
nds: of rej 
reinforcer: . P 
forcers, or reinforcement for two different responses may be 
- that schedules of reinforce- 


Progr; 
ammed si 
ment ae simultaneously. It should be clea 
= Je e . « 
enormously complicated. Ferster and Skinner (1957) wrote a 
nding found 


very 1; 

-7 targe book j : 

With diffe 3 book just to describe the different p 
rent kinds of schedules. 


atterns of respo 


he Theory 


Applications of t 
nforcement following the 


A 

LEStinn: 

Leann is P r P 
i eink is the procedure of withholding re! 
is first ia of a previously reinforced response. Jf a stable rate of responding 

stablished, then withholding reinforcement produces a characteristic 


atten 
in 3 = Fi ; : 
g out of the cumulative curve, own in Figure 7.3, Skinner 1s 


Usually Paes : i as sho £ . kin ‘ 
and he c i interested in producing behavior than 1n extinguishing it; 
lying A does not speculate about the mechanisms under- 
; inction. Is it due to loss of incentive motivation? Is it due to loss 
d S-R connection? Is it produced by 


In 
Stre; 3 
: ngth of the previously learne 
hese Hullian-flavored questions are never 
proceeds more 


Anhibiti 
a some type? But t ey 
. Skinnerians do ‘note, however, that extinction 
der others. In general, the less pre- 
slowly extinction 


s than un 
he base-line t 
is slower 
s reinforcement, a 


ask, the more 


following trai 
nd it iss 


rapi 
Pies Aes some condition 
Sine Rast annie ny is on t 
schede 5 example, extinction 
lna than following continuou 
(R. ng on a VI schedule. 
T E was defined 
NOS ing. But here we have 
If beh coments It is also assum 
avior collapses as a result of the ™ 


ning on an FI 
Jower still after 


nt that increases the rate of 
a somewhat different property of 
n a given rate of responding. 
ation of some consequence of 


earlier as an eve 


discovered 
ed to maintai 
ani pul 


130 Skinner 


t 
variable interval reinforcemen 
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fixed interval reinforcement 


continuous reinforcement 
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ction of respondin 
reinforcement e: 


Fig. 7.3 Extin 
kinds of prior 


g following three 
xperience. 


the response, then wes 


as re 
- p è ation has 
ire obliged to conclude this manipulation is here- 
. a . S > asis 
moved reinforcement It is ‘mportant to note the shift in emph 

For Hull, a reinforcer was ba 


i an event 
Sically a strengthener of connections, 4 
For Skinner, 


a reinforcer m 


ing, but 
ay produce So sia 
z ʻ . ayent 
amenta] property, Basically, a reinforcer is an Sve! ence 
x . iffer 
81Ves us contro] Over operant behavior. It makes relatively little diffe 
whether the behavior jis origin 


through re 
reinforcem 
have the I 


ally quite 

is simply m a constant rate an 
. * (e 

grammed conse of responding is foun 


inforcement or 
ent. If some pro: 
Property of m 


h 
strengt 
weak and brought up to strena 


with 


3 sarily be # 
8 behavior then it must necessari 7: of 
Teinforcer, We saw this logic applied to Skinner's original definition 
conditioned reinforcement, and it can h 


forcement, The old questi 


% s nary rét 
d as well to primary ! 
on that so fasci 
reinforcement is 


= 3 yer 
ascinated Tolman and Hull, en 
necessary for learning, is therefore answered logical Y 
rather than empirically, Jf the rate of re 
consequence of the 


$ be v some 
: 5 maintained by so 
response, then the conse, 


Nae Sequence must be a reinforcer. 
motivation recei 
(The principal 


The topic of y 
Skinner, reason for depri 
ensure that the Presentation of food y 
object is apparently to Produce some f; 


airly 
15 or 20 percent, and hold it a 


nies Severe leve] Of weight loss, like 
t i at thig Value, What is wanted is a level of 

deprivation that will be as high as Possible w 

debilitation of the animal, 


ių 
A few investigators h 


lout riskin 
upon rate of 


g the eventual 
ave looked systematically 
= k 
“sponding, and some of the results 


at the effects of deprivation 
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ween reinforcement schedules and depriva- 
a systematic relationship 
s and the strength 
he other hand, Powell (1971) 
7} schedules their motivation 
overall rate of responding, 
an FI schedule appears to 
nce with the schedule 


s 3 
e interactions bet 
between de a pean ea Clark (1958) has shown 
of hunger a ae rates of rats on different VI schedule 
found that wl Sone (see Figure 74). Ont 
seems to have ee pigeons are trained on \ 
or on the post ns e = any effect on scalloping, on 
be dependent Poin see pause. Behavior on ; 
and to be Sea entirely upon the pigeon's experie 
atively independent of its motivauon. 
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tivation and s inforcement. ThE variance 
under each co 
effect of depri 


ndition is 
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on variable-interval responding. Journ 
Analysis of Behavior, 1958, 1, 2: 
958 by the Society for the Experimental 


Inc.) 


cy of 
al of the Ex- 

$ 221-228. 
perimental 
Copyright 1 
Analysis of Behavior, 
addressed to these kinds of ques- 
y of how the rate of 
as motivation was 


as 
ported a stud 
a long session 
But he soon turned his attention 


d more interesting or more 


-Jy research W 
ation re 
ourse of 


: Some of Skinner's ear 
tons. For example, his dissert 
responding declined during the c 
decreased by the food that was consumed. 


to other determinants of behavior that seeme! 
predictable or that led to more stable responding across time. 

There is some question about whether oF not Skinner really has a theory 
of behavior. At one point, he explicitly denied having one (Skinner, 1950). 
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set 

that he does not have a theory if by that we ae to 
an underlying reality. Skinner has held stead 

vfulness of b : ans that 
‘ous system or mind; he pi 
awfulness found in gan a can 
a systematic orientation toward Liane to 
a particular bias toward what kind of pei only át 
der and describe them. Skinner typically looks : 


But theory can 
mean simply having 
make and how to or 


- Jong 
it can occur at very high rates over “ee 
at cumulative records. = ee 
nt to finding particular kinds a e 
ons about how behavior ae 
, and may they not have just as much Ia de by 
that are obtained as the prior Sapon eae 
in postulating a set of intervening varia bies? ae and 
Ptions about how behavior should be yagi ba 
rely made prior assumptions about how it shot 


» for example, 
makes prior assum 
analyzed. Hull mer 
explained, 


Skinner also tends to Overgener: 
tendency afflicts all of us from ti 
nostic of the theor 
retical. For exampl 
controlling hum 
tions, The human does not h 
order to be reinforced 
assumed to be conditio: 
tion, Skinner suggests 
social approv 
variety 


This 
alize on the basis of available data. Thi 
me to time, 
ist. Some of Skir 
e, he noted th 
an behavior 


but it is chronic in and na 
nner’s Seneralizations are clearly ee a 
rcers that are effective di- 
any specific motivating in 
ngry, thirsty, or frightenec arë 
social approval. These events jdi- 
ned reinforcers (a theoretical assertion), but, in adc Š 
» When a particular reinforcing event such as 

ablisheq asa Conditioned reinforcer under i 
comes independent A 
a generalized Conditioned reinforcer. 


; 
atement, one that is probably 
In explaining superstitious be 


food was presented at a 
mal’s behavior, the 
and rather bizarre beh 
scratch the floor, wave 


a) observed that, when 
dependently of the ani- 
reat deal of stereotyped 
aik around in circles, 
g the interya] between toe BB ae 
Skinner argued that whatever behavior an individual 
quite likely 


al in 
avior. 


presentations. 
happened to make 


increase in rate and be 
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angement in which the more often it 


and the more often it was 
tely each animal must 
none of which is neces- 
tingent upon any 
It is clear 


caug} í 
er 5 positive-feedback arr 
reinforce on eee it would be reinforced, 
develop ` K BAS likely it would occur. Ultima 
sary for a A of individualistic behavior, 
Particular beh; ng paul of course, because food is not con 

avior, Such behavior can be called “superstitious. 


that Ski 
3 Skinner has : 
like food iva s presupposed, as Thorndike did years before, that events 
an automatic response-strengthening effect, and he has pre- 
ith the contiguity between the 


suppos " 
sed fur K 5 
response seen that this effect increases W 

5 se and reinfor ar i 

basis ol eika ae Both these suppositions are defensible on the 
are clearly tl data, but in the context of the superstitious experiment they 

s y 1eore 7 . . 
y theoretical commitments and not just statements of fact. 


Still 

, the work j : ; A : 
e work of Skinner and his followers 1s relatively nontheoretical. 

their generally inductive attitude 


pirical accounts of their own 


Thei 
MeIr ci 
oncer 5 x ‘ 
ncern with nothing but behavior, 
1 consistent control of 


towa 
ard 
the accumulation of data, their em 


expe no 

riments x = s 5 

behavior T their preoccupation with reliable and 
, their emphasis upon empirical rather than conceptual resolutions 


Of pr 
il all indicate that Skinner's theory is the least theoretical of the 
a clear ane Me have examined. Furthermore, over the years there has been 
OP Organis c ayay from learning theory in Skinner's writing. The Belau 
A A (1938) was obviously a contribution to learning theory. It 
Methodol much of the author's early thinking related to philosophical and 
contril Q ogical questions. It contained a wealth of data too, but its main 
in sin publoniat the time was to the better understanding of animal learning 
ee ste situations. In effect, the theoretical questions in animal behavior 
Seale Pech solved, at least by common consent among Skinner's followers; 
EAN ana A then turned to the complex problem of applying their 
ds to human behavior. Here the behavior was more complex, the 
data were sparse, and there was renewed freedom for theoretical develop- 


ments ; 

p (for example, Skinner, 1953). 
a t is also apparent that over the y 
Impler to more complex organisms, 


cage practical questions, from labora 
Ie has made a “flight from the laborator : $ 
an ever-increasing number of 


5 have been applied to 
s shown, accordingly, less interest 


and Skinner has 
and more interest in the maintenance of be- 


behaviors and new kinds of stimulus control is 
ng run than simply regulating and main- 
lling) behavior. To the extent that current em- 
s of behavior-control techniques, there has been a 
learnin mechanisms oy SC; 

g ms per se; to that extent 


not only moved from 
he has also moved from theoretical to 
tory situations to real life situations. 
y” (Skinner, 1961). Operant-condi- 


ears Skinner has 


tioning principle 
human-behavior situations, 
lem of learning 
new 
nt in the lo 


in the prob 
havior. Acquisition of 
viewed as less importa 
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lication 
with 


taining (in shor 
phasis is upon apP 
lessening of concern 
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ess theoretical and, of course, s 
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a scientist: practical rather than ‘Aes so 
me idea of how to control a particular AnA 
it works. If it does not work, then we must se 


technique that is more effective, 
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really all we 


ment schedules. Fixe. 
high can the r: 
192 reported b 


Soon after dey the methods and a conceptual framework for guar 
ing behavior Systematically į si aboratory situations, Skinner ane 
his atte matters, One Was the training Of animals for suc 1 
i ectors and Missile pilots (Skinner, 1960); 
San working with Pigeons, Skinner discovered that they 
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general ar 
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Skinner’s Impact 
In recent years Skinner has had a tremendous influe 
the 1960s was probably even greater than Hull's w 


‘ à Bich is his 

Several aspects of his approach have been particularly appealing. One ime 
a é $ oi Cd 

atheoretica] empha During the 1960s a number of psychologists be 


: l $ atata s were 
disenchanted with theory in general. Hull's own theoretical statement 
not well supported empirically, 


little success in patching u 
there gradually developed 
wrong with Hull's t 
quantification or 
the fundament 


. . k win 
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control everything to 
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the 1930s. The most imp 
learning experiment and 
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in the 1920s. 


minat ut the Hullian hordes were 
4 a H ] ri 
band of followers took charge in the 1960s. Whether they aa © 1950s. Skinner and his 
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of behavior that 
rprising that 


of the earli 
approach act pet a had turned out 
after all, the 1e practitioner immediately in ¢ 
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f interest in reinforcement tech- 
agencies—everyone it 
and control the 


ultima 
there sai ih to the practitioner. 
niques, oh oped an immense amount © 
seems—is ee parents, law-enforcement 
behavior OF thei te use reinforcement techniques to shape 
useful bites charges. It is well past time that psychologists made a 
great credit to Bese to the practical management of behavior, and it is a 
cedures that a miner that he has provided concepts, techniques, and pro- 
Much of aioe to be more useful than those of any of his predecessors. 
Skinner's success is attributable to other men devoting them- 
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s, for 

X one rea = ie š 
e reason or another, to his cause. Skinner and his students at 

ind 1950 by a powerful group of allies 
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rant-conditioning Jabora- 
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at Geir were joined arou 
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try, ; 
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y had a group of very able students who app 
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the land. 
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1938) is still of considerable value, even 
dated. Schedules of Reinforcement (Ferster & 
e amount of material on schedule effects. The 


js Rachlin (1970). Skinner is among the 


Th j n 
e Behavior of Organisms (Skinner, 


tho 

eked: 5 of the material is now 

best TE 7) contains an oppressive a! 

Lest of account of operant conditioning 3 - Á Š 

tötéai contemporary writers, and his Cumulative Record (Skinner 1972) is a joy 
, as well as being highly info of the papers cited in the chapter 


are repri rmative. Most 
eprinted there. It tells us much about th vel] as his many intellectual 


a e man as W 
Aventures. 


PART 


TRE 
CONTEMPORARY 
SCENE 


In Part I we emphasized the differences among theoretical positions in 
order to stress their distinctive features and to focus on certain key theo- 
retical issues. But history does not leap about; it flows. There has been 
continuity, there have been overall trends, and there has been a gradual 
accumulation of understanding about learning. In Part II the focus will be 
on the contemporary scene. We will try to make explicit some of these 
overall trends and to see if there is some consensus among contemporary 
views of learning. In Chapter 8 we shall focus primarily on different views 
of classical conditioning and on the important question of how classical 
conditioning interacts with instrumental, or operant, learning. We shall see 
that, although there is a working consensus on these matters, there are also 
some very difficult theoretical problems that remain to be solved. In Chapters 
9 and 10 we will conclude our an 


alysis of learning by pointing out some of 
the directions in which contemporary theory seems to be moving. We will 
see that contemporary approac 


hes to learning have been enriched, first, by 
the inclusion of broad evolutionary P 


rinciples and, second, by the revival of 
cognitive conceptions of animal behavior. 
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CHAPTER 
8 


THE STATUS QUO 


al learning theorists have tended to pull us con- 


ceptually this way and that, there have been some gradual overall trends 
and continuities. Some of these continuities may be apparent to the dis- 
cerning reader, but others must surely be difficult to perceive. For example 
the concept of classical conditioning was first promoted by Pavlov then 
discounted by Thorndike, altered by Watson and Guthrie, minimi b 

Tolman and Hull, and acce: ted by Skinner. The reader would have to va 
discerning indeed to perceive any trend here or to draw any conclusio 

about the current status of classical conditioning. In this chapter Sia i 
make current thought about classical conditioning more explicit. W = 
see how(classical conditioning and instrumental learning together kone 
the main framework for learning theories in the mid-1970s. er constitute 


Although our classic. 
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Basic Paradigms 


Earlier chapters have revealed a diversity of views on the feortanents” 
nature of the learning process.(Thorndike and Hull proposed that lear — 
depends upon a reinforcement mechanism, Guthrie proposed that i 
tiguity is sufficient, and Tolman put forth a quite different proposi) : 
big question—What is the fundamental nature of the learning po j 
as unsettled today as it has ever been. There is, however, less concern today 
than there used to be about this ultimate question. One r j Ti 
of concern may be a growing skepticism about whether the question Ci 
ever be answered. A second and perh f 
even though there is little agreement about the basic processes of —— 
a good agreement about the procedures that produce learning has grown up- 
The procedures are relatively clear and useful, w 
uncertain and rather conjectural, an 

The consensus is that ithere are two effective experimental nmin 
two basic paradigms, for producing learning, The first is that discovered N 
Pavlov. )It can be called P sical conditioning, O 
respondent conditioning, According to this paradigm, Tearning occurs when- 
evër a relatively unimportant stimulus § is paired with an ce pein 
stimulus $*, such as food or shock. he second paradigm is variously calle a 
trial-and-error learning, instrumental learning, reinforcement learning, O! 
Operant conditioning. According to this paradigm, learning occurs when oe 
important event §* is made contingent upon a particular response, Because 
of the unanimity about the effectiveness of these procedures, the terms 
“operant conditioning” and “classical conditioning” have tended to become 
labels for the procedures themselves, rather than designations for the cor- 
responding but still unknown learning Processes. This relabeling is not 
altogether happy from a theoretical point of view, but it is workable enough 
for the empiricist because he can proceed with his experimental study of 


learning. He can study as thoroughly as he wishes the conc 
which learning is found, 


This methodological, practical, 
raises a number of questions, whic 
seeking the ultimate basis of learning, are nonethele 
limited sense of seeking general answers to methodo 
questions. What type of responses are learned with 
dures and what type of responses are learned w 
procedures? What is the critical procedura] 
The CS and the US are always paired, but 
ing per se or upon some other kind of rela 
US? In an operant conditioning procedu 


son for this loss 


: a is that 
aps more important reason 15 that, 


sec FONN 
hereas the processes rema 


‘avlovian conditioning, cl 


litions under 


Or Operational] appro: 


. ach to learning also 
h, if not theoretical i 


n the grand sense of 
ss theoretical in the 
logical and practical 
reinforcement proce- 
ith classic 
aspect of cl 
does learnin, 
tionship bet 
re, the rein 


al conditioning 
assica] conditioning? 
8 depend upon pair- 
Ween the CS and the 
forcer is always con- 
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arning result from this contingency 
? To what 
simultane- 


tingent upon the response, but does le 
or fx tle x 

r from some other aspect of the operant-conditioning paradigm 
extent do classical conditioning and operant conditioning occur 


ously in a given learning experiment? For example, if we attempt to re- 


inforce a particular behavior, may We not also be inadvertently classically 


conditioning some other response? 
In the present chapter we will attempt to summarize current thinking 
about these kinds of issues. We will grant, for the moment, that there are 
two basic paradigms. We will see how precisely these paradigms can be 
defined and how much we can find out about them. Then we will be better 
prepared to return to the big question of the ultimate nature of learning. 


Two Response Systems 


(During the era of Watson and Guthrie, all learning was said to be classi- 
cal conditioning; but}the work of Tolman, Hull, and others clearly indicated 
that most responses are not readily classically conditionable. It was there- 
fore suggested by(Skinner (1938) and argued at some length by Mowrer 
(1947)(that classical conditioning was applicable only to certain behaviors, ) 

1 respondents, such as glandular secre- 


nner called 
omic nervous system. (he learning of be- 


and the like was assumed to require 
onses subject to operant con- 
to respondent or classical 


a class of responses that Ski 
tions and reactions of the auton 
haviors such as running, bar pressing, 
reinforcement) In short, we had operant resp 
ditioning and respondent behaviors subject 
conditioning. 

This comfortable 
to some of the earlier Hullians. Hull 


take a classical-conditioning experimen ‘ , 
that the dog did not salivate to the bell because the ensuing food pellet 


reinforced salivation. The same argument was applied to fear conditioning, 
most notably by Miller (1951). Tf shock is the US it must ultimately termi- 
nate, and, when it does, there has to be a large reduction in drive that should 
be effective not only in reinforcing operant behavior but also in reinforcing 
the fear reaction itself. In effect, the rat learns to fear the CS in much the 
an learns to bang his head against a wall: because it feels 
so good when it stops. In discussing this possibility Miller (1951) noted that 
it is contrary to common sense, but he also pointed out that common sense 
often proves to be wrong. Mowrer conducted a series of experiments to 
defend the idea of two basic learning paradigms by attempting to separate 
them experimentally. In a typical experiment, Mowrer and Solomon (1954) 
found that the strength of fear conditioning depended not upon the relation- 
ship between the CS and the termination of the US, as Hull and Miller 


ering of things was not acceptable 
(1943) noted that if we were to under- 
t on salivation we could not be sure 


and convenient ord 


same way that am 
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argued, but seemed to de 
onset of the CS and the 
It was apparently the co 
fear reaction and not th 


pend instead upon the relationship between ans 
onset of US, as the Pavlovian paradigm one 
ntiguity of these stimulus events that a 
€ subsequent relief from shock. Miller (1963) s 


á 2 eo ction, but 
sequently abandoned the idea that fear is learned by drive reductio 


‘ y s oe aa -esponses 
he still did not accept the idea that there are distinct classes of re l 
that are ruled by distinct le 


arning principles. His tactic was to P 
drive reduction for both operant conditioning and classical condition a 
which took a heavy burden off the drive-reduction hypothesis but left Pi 
the question of whether or not the two types of conditioning are uniqu 
applicable to the two kinds of behavior, ; E 
The story does not end here, however, For several years Miller joi nse 
tinued to investigate whether or not the distinction between the two meape oa 
classes is valid. In the first reported study of this series, Miller and fervor! 
(1967) conditioned salivation using operant techniques. They had a0 jë 
that during the course of a session there was spontaneous variation in oe 
amount of salivation: so they simply made water reinforcement continget 
upon the occurrence of a relatively high rate of salivation (water was > 5 
instead of food to eliminate the possibility that the reaction was mere e 
classically conditioned response). A companion group, necessary to coner 1 
actors, was reinforced whenever it gave a lowered 


for various extraneous f. trol 
. : . . S n 
rate of salivation. In both cases salivation was found to be subject to cor 


by its consequences. 
In another experiment DiC 

to avoid an electric shock by i 

is not remarkable because heart- ase might be interpreted as 4 

classically conditioned pe 

interpretation, DiCara 


ara and Miller (1968) 


ncreasin 


€ experiment was 
the rats were deeply para i 


curare-type q a similar study 
Miller's student Trowill (1967) trainea EP drug, In a similar 


: Curarized rats either to increase or 
to decrease their hear 


ical brain stimulation. 
s, whicl í Miller, 1969) indi- 
: Ean d to be involuntary 


8astric motility, the 
body, and localized 


Two Response Systems 145 


Pel si 


JERR S Hath 
=S 


400+ 


Heart rate (beats per minute) 


slow 


300+ oe SS ee eae 


25-Trial blocks 
rate in rats reinforced 


Fig. 8.1 Changes in heart 
for faster or slower rates. Results are given for the 


response to S+ signaling shock, S— signaling no 
shock, and “blank trials” on which no cues were 
presented. The results indicate large overall changes 
plus smaller changes specific to the cues. (From 


DiCara & Miller, Changes in heart rate instrumen- 
zed rats as avoidance te- 


tally learned by curari: 
sponses, Journal of Comparative and Physiological 
Psychology, 56, 1968, 8-12. Copyright 1968 by the 
American Psychological Association. Reprinted by 
permission.) 

There is an important practical application for this kind of training in 
psychosomatic medicine. It is possible that people can control their own 


blood pressure or heart rate, once they learn how. The argument is that 
such internalized responses are not commonly reinforced in our culture 
because they do not ordinarily affect anyone else. The community reinforces 
our verbal behavior and our overt movements because such behaviors affect 
them, but no one really cares about our internal emotional reactions, so 


they are not reinforced. Therefore, according to Miller, a myth has grown up 
that such behaviors are not subject to reinforcement. 
ller and his colleagues has a number of 


_ Although this recent work by Mi 

interesting theoretical and practical implications, the only conclusion that 

we can draw in the present d ion is that while we may continue to 
asses of learnable re- 


s iscussi 
think of two distinct paradigms and two separate cl 

sponses, these two dichotomies can by no means be put into a one-to-one 
correspondence. The two dichotomies do not overlap perfectly. We cannot 
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conclude that, just because responses like he 
ment, they are not also subject to some kir 
lovian learning process. The fact is th 
changes in an organism it is much mor 
and then use the CS to control the res} 


by means of the Operant paradigm. Even though operant methods de por 
(see Figure 8.1), the changes in heart rate do not come about quickly. Pa 
effects achieved by Miller and others often require tedious and teclimica a 
difficult procedures which contrast sharply with the speed and sana oe A 
comparable effects that can be produced with Pavlovian procedures. i si 
there is good reason for continuing to believe that the Pavlovian peas ie 
if not uniquely applicable to autonomic responses, is at least pino! n oe 
effective with such responses. In effect, we still have two kinds of learning, 
we still have two kinds of responses, 
Respondents are learned with res 


Operant conditioning. For m 
good. 


art rate are subject to reinforce- 
nd of nonreinforcement or Pav- 
at if we wish to control o 
re efficient to pair a CS with hte 
ponse than it is to establish contro 


é + ast mb: 
and we still have a good rule of thu a 
. e . 39° wil 
pondent conditioning and eae ids 
$ . sg sti oO 
ost purposes this conventional rule still h 


Two-Factor Theory 


The two learning paradigms inter: 
very important. Suppose a r 
assume that the food pellet 


may also assume that 
digm, 


. q 
act in another way that is theoretically 
at has been trained to press a bar for food. We 
reinforces and maint 
» in accordance w 
the food pellet is a US that cl 
salivation to all the stimuli pre 
should be an interaction bet 
cause, whenever we conduct 
ent conditioning to occur simultaneously, 
Hull's secondary learning system was b 
as we have already seen. Rein 
to the strength of habit, a 


ains bar pressing. But we 
ith the classical-conditioning para- 
assically conditions responses such as 
sent in the Skinner box. In other words, there 


à -s be- 
ween operant and respondent procedures b 


an operant experiment, we can expect respond- 


ased upon just such an interaction, 


Behavior strength = K XD X ati 


Here the habit is assumed to be moti 
D and the secondary motivator 
strongly the unconditioned consu 
box and how strongly the conditioned response ro 
those in the goal box. As Hull’s secondary learni 
by Spence and his students, it provided a clear conc 


y both the prim 
Strength 
mmatory response js 


ary motivator 
depends upon how 
elicited in the goal 
is elicited by stimuli like 
ng system was developed 
€ptual Separation between 


K, whose 
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operan y arne abi i i i 
l tly learned habit and Pavlovian learned motivation. We saw in 


Chapter 2 i š 
pter 6 that this model was used to explain phenomena, such as latent 
Hull's primary learning system 


learning a i fo: 
irning, that were extremely difficult g 
1 in the goal box 


pays for. When the rat suddenly encountered fooc 
Na Sa ii To was sufficiently conditioned to provide substantial 
a or the learned running response. 

If WA think of sH, and K as acquired through independent kinds of 
learning—by the two different basic paradigms—then it ought to be possible 
to manipulate gH, and K independently. More specifically, it ought to be 
possible to demonstrate the independent acquisition of the habit factor and 
the motivation factor in a latent-learning situation somewhat simpler than 
the complex mazes used by Tolman and his students. An attempt to do this 
1949).! Seward first let animals explore a T maze 
o food was present in the maze at this time). 
d in one of the goal boxes with food. When 
the animals were then placed in the start box and allowed a single test trial, 
them chose that side of the T maze where food 
hink in cognitive terms (Tolman had inter- 
ying that the animals had learned where 
ing placed there) or in Hullian 
here the animal was given 
al will approach the pre- 


was reported by Seward ( 
and become habituated to it (n 
Then they were passively place! 


a significant proportion of 
had been located. Whether we t 
preted a similar experiment by s 
the food was in the apparatus simply by be 
cues like those present W 
liction that the anim 
1 this is what Seward found. 

ies with this kind of experimental proce- 


dure, however. For example, it ought to be possible to show the same facili- 
tation of performance in a simple runway following direct placement ina 
baited goal box. But this effect is evidently extremely difficult to obtain 
(Seward, Datel & Levy, 1952), and even the latent-learning phenomenon in 
the T maze does not always come out as theory predicts. Seward and Levy 
(1949), however, reported a technique that more effectively supports an rg 
interpretation of instrumental performance. They reasoned that if an animal 
had first learned to perform in the apparatus until behavior had stabilized 


and then was given direct placement in an empty goal box, this experience 
should produce immediate deterioration of instrumental performance: a 
phenomenon aptly named latent extinction. The argument is that, just as 
Ta can become conditioned to goal-box cues independently of the running 

nguished independently of 


this conditioning can be extir 
ehavior. Although the original latent-learning procedure 


terms (rg is elicited by 
food), there is a clear prec 
viously positive goal box, anc 

There are some peculiar difficult 


response itself, so 
the instrumental b 
d is an important theorist in his own right. Quite independently of 
wrote during the 1940s and 1950s a series of provocative theoretical 
loped his own version of a secondary learning system, emphasizing 

which motivates and directs behavior. 


1 John P. Sewar 


the Hullian group, he 
he deve! 
fa mediating reaction like Te. 


vides a good summary of his thinking along these lines. 


papers in which 
the importance © 
Seward (1956) pro 
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ith and rather unreliable, the latent- 
be much more effective and reliable, 
ore widely investigated, : re 
ot entirely free of interpretive ge 
S the effect is not found (Koppman 8 
is found, it is not facilitated by i 
cen goal-box cues and cues elsewhere in th 
hich it should be according to the ro interpretation. PTE 
difficulties, there has been growing ne 
vation can be manip 
it is supposedly motivating. One reason any 
interactive effects of Pavlovian and reinforce™ nt- 
emonstrated in a variety of situations besides o : 
inction experiments. One of the newest oie nen 
Procedure reported many years ago by Estes. W he 
as published in 1943, jt attracted little attention, but i 
que has been widely used in recent years, Estes waa aie 
20X to work for food; then the bar was withdrawn sie By 
given a series of Pavlovian Pairings of a buzzer followed 
animal was retu 


apparatus—w 


Estes’ Study w. 
general technj 
in a Skinner | 
animals were 


as a probe on the eet 
hen the buzzer was Pa 
bar pressing during extinction. Aga d 
us that has been paired with food can have a pronon e 
formance of an instrumenta] task, appearing to seer 
» even though the cue to food signifies nothing in 

base-line task. It Signifies food only in the Situation in which it was estab- 
lished, Nonetheless there ; > and we m 


a dramatic rise in 


a ski 5 rained 
and Skinner (1958) tra i 
IET . ; wer 
ase line colored lights w 
2 Animals are usually tested du in incti, "a 
as KANE PNE extinction of the base-line response because, first, 
this ensures that the cue is never paired With Teinforcemen pons ieee and 
i i i A a e-]] ASK i 
second, bigger effects are found in extinction (Bolles & Sia a the base-line ipualy 
learned response is a free Operant, for example, bar Pressin a 70). When the ne TE: 
probe; it is simply intruded into the Situation for a few eee cue is quite Hee aa 
its effect on the rate of the base-line respon, Heic S or minutes to see cs 
“probing the base line, Estes Procedure, like the din aR Such recent expressio! Jn 
i ip between $ Placement Procedure describe 
is, “i i the Cs a inden » of 
base-line behavior, that is, “off the base line,” Comparable Sinan, food independently b: 
also be done “on the base line,” simultancously with the toning could, of course, 


asic rein forcement schedule. 
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dis j 
isplayed on the key. One color signified free food which could be obtained 
another color signified no food. Then the 


wi ; 
without pecking the key, whereas 
g was extinguished, but 


elon ae ae to the base-line task and peckin 
one Henn wee T different key lights were introduced as probes. Morse 
Selene tor found that the key color that had been associated with food 
i aintain pecking, whereas the key color associated with no food 
suppressed pecking and facilitated extinction. 
number of experiments showing further the 
between Pavlovian training and 
(Trapold, 1966), he 
at in the presence 


7 Trapold has reported a 
P calls it transfer of control) ¢ 
ental performance. In a typical experiment 
discriminative control so th 
reinforced, whereas in the presence of 
as extinguished. After the dis- 
animals. For the controls 
a series of Pavlovian 
tion learning and the 
a signal for no food and the S* 
Pavlovian trials the reversed 
consequences occurred independently of 
ase line. Discrimination reversal pro- 
animals that had the 
1 it did for the controls. In another 
and Myers (1965) examined the temporal 
] scallop. Rats were first trained non- 


contingently with 300 free food presentations, on a fixed two-minute 
schedule. Control animals received 300 noncontingent free food pellets on a 
variable two-minute schedule. At all animals (which had earlier 
learned to press the bar for continuous re ent) were trained on an 
F1-2 reinforcement schedule. It was discover’ animals that had 
been given free food at a two-minute interva 

the fixed-interval scallop much more quickly t 
be supposed that during the noncontingent, 
amount of time elapsed from the 1 reinforcement be 
and as an incentive motivator, whi ated bar pressing when 
the animals were shifted to the hase-line task. 

Most of the studies just discussed have been described by their authors 
operationally; they emphasize the novel procedures and minimize the theo- 
retical interpretation. Specifically, an re motivational interpretation is rarely 
offered. There is, however, another very large body of re rch that draws 
very heavily upon conventional rg theory and lends considerable credence to 
it. This research is Amsel's work on frustration. It is not our purpose to 
survey all the research on frustr to draw any conclusions about 


brought bar pressing under 
of one stimulus, $2, responding was 
S3, responding W 
as reversed for all 
animals 


an alternative stimulus, 
D ete 3 ; 

imination was established, it w 
experimental 


it was si 
was simply reversed; for the 
he original discrimina 


trials was i 
Us was interposed between t 
LEVATEA] fratan i x 

versal training in which the S? became 


beca ` i 5 
ecame a signal for free food. During these 


associati s 4 
ociation of the cues with their 


and off the ba 


the animal's behavior 
the experimental 


ceeded much more rapidly for 
reversal experience off the base line thar 
experiment Trapold, Carlson, 
control involved in the fixed-interva 


this point 
inforcem 
ed that the 
] off the base line developed 
han the controls did. It could 
or Pavlovian, sessions the 
ast gan to serve as an sp 
ch then gener: 


ation or even 
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what it portends for the concept of incentive motivation. But Amsel’s (1962) 
theory must be mentioned because it constitutes 
the rg concept. Amsel’s b 
by cues in the situation a 
an emotional reaction 
emotional state sery 


appetitive rg is assu 


an important extension a 
asic premise is that, whenever r has been ara 
nd then reinforcement fails to occur, there we 
(presumably something akin to anger) and that 
€s a motivational function very much as fear or = 
med to do. Amsel thus has a pair of parallel a 
tional agencies, one, the conventional ro, which is the conditionable a 
of the consummatory response, and a new one that is the Ta z 
part of the anger or frustration response (which is symbolized rp). Tliis i 
is hypothesized to produce withdrawal and the inhibition of O bë 
much the same way that rg produces approach and the facilitation O ved 
havior. In a discrimination task, according to Amsel (1962), re oe 
conditioned to the positive SP, whereas rp becomes conditioned w S$. 
crimination learning can be mediated by either or both ee eN y 
These complimentary mechanisms permit us to do a lot of explanato 

tricks. Consider, for example, animals tr 

ment. On nonreinforced tria 
cues. But there must also be 
follows nonreinforcement. O 
responding, even though rp 
occurred a number of times ( 


ained with intermittent aero 
Is rp must become conditioned to tanec 
a number of occasions on which reinforcement 
n these occasions the animal is reinforced ior 
tends to inhibit responding. After this me 
Amsel, 1962, describes evidence indicating = 
it must occur on the order of sixty times), then ry will no longer be a meee 
tive factor; it will become part of the total SP configuration in the presence 
of which the animal is reinforced. At this point the animal will continue to 
run, press the bar, or whatever, even though it is frustrated at the time it 


does so. Finally, if the animal learns to respond when it is frustrated, then 
it should not be disrupted by extinction, If 


continuous reinforcement, sudde 
tion of responding; 
the presence of Tp, no 
animal should continue to respond. In shor 
the partial-reinforcement extinction effect, 
tion had caused a great deal of embarras 
Hullians had been quite unable to cope 

with it only by assuming that it was a 


, irical law of such generality that 
it did not have to be derived from more basi s 
a derivation. 


s inèd on 
an animal has been — 
ation should produce rapid inhibi- 
as been reinforce 


n frustr 
but if an animal h 
extinction presents 


Let us consider one further F msel’s —_— 
It is known from a variety of ex ; 
Crespi (1942), that rats run mor 


and in general perform better if 


r mor ently, 
they recei e frequ y 


ve a large amount of reinforce- 
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a small amount. Intuition suggests that this 


superiori 
Faerie ee ei task should carry over into extinction, so that 
ware resistin sini received a larger amount of reinforcement should be 
trary, there sho Apap But Amsel (1962) argues that, quite the con- 
ete ce deat a . bea Terma in much the same way—but for a different 
Bins whi, sccn ergs a reversal following partial reinforcement. He argues 
fonar ne vernioreemens there is more conditioning of rg or condi- 
ben E et vigorous rg and that when frustration occurs there should 
etadonae p N stronger Tr leading to greater inhibition of the learned 
Gove atid D ne! words, a larger amount of reinforcement should lead to 
tation E extinction when food is withheld because of the greater frus- 
Gainers ha entails. Furthermore, this reversal should occur following 
contesits 3 te not intermittent reinforcement training. ei well- 
1961) d studies show that this 1s indeed the case (Hulse, 1958; Wagner, 
Amsel’s frustration theory is no 


A constraints upon the generality 
Rieti las analyzed. For example, the anomalous a ofa 
Brelsf n extinction appears to occur only after extended training (Theios & 

sford, 1964) and only when animals are tested while severely deprived 
(Marx, 1967). Amsel’s interpretation of the partial-reinforcement effect has 


b ý 4 ve : 
een sharply criticized on both theoretical and empirical grounds by Capaldi 
Amsel’s work on frustration has con- 


(1970), among others. Nonetheless 4 

siderably strengthened and extended the secondary-learning system originally 

proposed by Hull and Spence, and it has contributed substantially to the 
l behavior is in part con- 


general acceptance of the idea that instrumenta 
conditioned mediating responses 


trolled and in part motivated by classically 


such as rg and rp. 


ment than if they receive 


t without its own problems. There appear 
of some of the phenomena that 
mount-of-reinforcement 
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and his students, discuss e, Tr» and 
as if they were real responses. Other 
Estes, and Trapold, make no specific reference 


d response; they talk about pairing a stimulus 
ments simply in these operational terms. 


This latter strategy (or personal style) is operational and descriptive; it is 
also conservative and safe. But consider the enormous advantage that would 
result if it were possible to observe Ta directly. If we could monitor Te, 
perhaps by measuring salivation from moment to moment in the hungry 


xample, Spence 


Some theorists, for € 
oned mediators 


other classically conditi 
theorists, including Skinner, 
to any classically conditione 
with food and describe their experi! 
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rat, we would be able to test 
tem. Furthermore, 
salivation, w 


empirically the whole secondary Leating be 
if it could be shown that the rg for hunger really en 
e could then apply the entire body of research producec Me 
Pavlov and his followers to the explanation of incentive CAN ean es 
would then be in an excellent position to verify and expand upon a r 


: is 
; Siae - oe li remain at this 
ber of implications of the secondary learning system which rema 
point largely conjectural. 


It should therefore not be 
attempted to search for 
the stomach, or in the 


surprising that a number of investigators ie 
the real re, to find whether it is in the Ta pie 
unfathomable depths of the central nervous sys 

Some of this research has monitored salivation 
Motivation of instrumental behavior; 
Motivational effects fo 
salivation, None of th 


Support the conclusion 


simultaneously with ae 
some of it has looked for the oe 
lowing the administration of drugs that anon 
is research has been very encouraging. It does ne 
that the hungry animal's ro is located in the pen 
Some of the evidence Suggests this, but it is far from conclusive. A dee 
Study of thirsty rats was reported by Miller and DeBold (1965). 
measured minute tongue movements of thirsty 
that incipient tongue movements might be 
consummatory response, Agi 


animals on the assumption 
a conditionable part of os 
ain the results were provocative and lent aioe 
Support to the idea that To had really been located in the thirsty rat, sh 
the evidence was far from conclusive and has not been confirmed by st 
Sequent research. 

So the locus of Ig 
whether rg has an 


question, Rescorla 


remains uncertain: 
y reality status at all, 
and Solomon (1967) co 
tion for believing in the reality of rg. 
is a fiction because the concerted sear 
investigators over a number of years h 


because, if we assume that Somewhere there is a classically conditioned 
response, we can then attribute to it all the known properties of classically 


conditioned responses. But perhaps this Convenience js illusory: What do 
we really know about classical conditioning? Rescorla i 
clude that it is only the Pavlovian procedur, 
number of Pavlovian trials in which ey, 
of the animal's behavior, 
sequent instrumental beh 


P p 3 -ertain 
indeed, it remains uncer z is 

Bee w of this 
In their important review of 


eS justifica- 
nclude that there is little justifi 
It is, at best, 


$ i It 
a convenient fiction. 
ch for it by 


r wary able 
a number of very a ai 
as failed to reveal it; it is convenie 


and Solomon con- 
üna 

© that matters, If we run 

A | events are Programmed independently 

this experience can have a marked effect on sub- 


i avior. Whatever it is that happens in the Pavlovian 
part of the experiment, it is the noncontingent Presentation ef cuesiand 


consequences, either on or off the base line, that matters 
some mediating response becomes attached to the cue, T 
way, the paradigm retains its value even though koe 
conditioned cannot be confirmed. 


and not whether 
© put it another 
N€ory of what is 
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Classically Conditioned Fear 
applied to aversive behavior as 


me = ean learning system is often i 
abie ina eae behavior and many of the same techniques are appli- 
weap x ne first study of classical-instrumental interaction was the 
EANN o aie and Skinner (1941). 
schedule. ae irst trained to press a bar for food on an FI four-minute 
upon ie 4 a this behavior had been stabilized, there was superimposed 
aT ne ine task a series of cones of three minutes duration that were 
One: plomen by shock. Initially the tone alone produced no suppres- 
ih Sie pian with the base-line task, but after about eight pairings 

ak and shock the tone produced virtually total suppression of bar 
Pressing. The argument is that there was classical conditioning of a fear 
reaction to the tone (established in this instance on the base line) and that 
fhis emotional behavior effectively interfered with and thus suppressed bar 
ieee This general procedure, which has been called the “conditioned 
ssion” and the “conditioned emotional reaction (CER) procedure, 
and powerful method for investigating fear 
at the effects on behavior are usually pro- 
conditioned in very few 


experi ment of Estes 


pees a very popular i 
leuna iRing Its great virtue is th 
mp and easily measured, and fear can be 
avlovian pairings. 
originally described by Mowrer and Solomon 
litioned punishment.” Mowrer and Solomon 
food reinforcement and then pre- 
1 shocks, the purpose of which 
tone. After the tone had been 
duced, not as a probe, but as a 
henever the animal pressed 
was markedly suppressed, 
luced. Thus, we see two 
behavior when the 


P Eria procedure is that 
first 5 which we may call “conari l 
a rained animals on bar pressing with 
sented off the base line a series of tones an4 
Was again to classically condition fear to the 
associated with shock a few times it was intro 
consequence of the instrumental response. Now w 
the bar, it produced the tone briefly. Bar pressing 
or punished, presumably by the fear that it proc 


effects of fear conditioning. One is to suppress ongoing i 
CS is introduced as a probe, and the second is tO punish ongoing behavior 


when the CS is made contingent upon it. : 
There is still a third way in which fear is assumed to affect instrumental 
behavior, and that is in the avoidance-learning situation. When the sec- 
ondary learning system was frst described in Chapter 6, we noted that 
Mowrer and Miller had contributed greatly to its popularity by suggesting 
was to show that fear is 


oted earlier, 


nt, as we n 
gain here to illustrate Mowrer and 


ime! 
ent is cited a 
nditioning of fear. 


The main purpose of this € «peri 
classically conditioned. The experim 
Solomon's novel method of assessing the co! 
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that fear could serve as a learned source of drive. The argument is that a 
becomes classically conditioned to the CS (the warning stimulus) in ae 
avoidance situation and that this fear serves two functions. One is to; i i 
vate avoidance behavior, and the second is to set up the possibility o 
subsequent fear reduction will reinforce the avoidance BESO teal 
view of avoidance behavior was first spelled out in an important vie of 
paper by Mowrer (1939) and then tested in a series of ingenious aie gk 
which we shall describe just one. Mowrer and Lamoreaux (1942) HOLS Ethe 
if fear becomes classically conditioned to the CS, then termination p ee 
CS should serve as an effective source of reinforcement for BUDS ad 
havior because it should be accompanied by a rapid reduction in seat s a 
a concomitant reduction in drive. Mowrer and Lamoreaux ran three B! e 
of rats in a shuttle-box avoidance task. The groups differed in the A 
ship between the avoidance response and termination of the CS. One A os 
received a very short CS in a trace-conditioning paradigm in which i ae 
came on momentarily and went off some seconds before shock was sae 
to occur, An avoidance response could avoid the scheduled shock put h 
not affect the brief CS. A second group received a very long CS, aia 
persisted after the shock was scheduled to occur, and even if the anii 


? z À J ; afterward, 
made an avoidance response the CS persisted for some seconds afte 
So that for this group too the CS y 


third sroup received a CS that w 
animal made an avoid 


x A se, The 
as not terminated by the response. fn 
aS programmed to terminate whenever 

ance response, 


as just right. Mowrer con- 
ith the termination of the 
for the avoidance response. 
O groups was attributed to anomie’ 
Parasitic reinforcement, not inyolving 
together with further results reported by 


h sibility to the argument that avoidance be- 
havior is mediated by a classically conditioned fear reaction. The picture 


we had was that the fear motivates avoidance behavior at the onset of the 
CS and reinforces avoidance behavior when it is reduced at the termination 
of the CS. In the context of motivation, fear is usually called an “acquired 


drive,” but its characteristics are very much like those of incentive motiva” 
tion. Fear is assumed to be a classically 


$ 5 š Conditioned emotional response, 
which is sometimes symbolized Tp, and it Serves the same kind of mousi 
tional role that rg and Tr do. It is an important, indeed a ee art of 
the secondary-learning and Motivating system, for within (ee coe nde 
no more possible to explain avoidance w ys 


it ah et A 
hout Tr than it is to explain latent 


e of reinforcement 
The slight learning shown in the other tw 


mechanism, which Mowrer called 
fear reduction. This study, 
Mowrer, lent tremendous plau 
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learni 
arnin s = Š s A S 7 
illustr 5 or the Crespi effect without rę. The avoidance-learning experiment 
strates " - a y s 
conditi es the same kind of interaction between a hypothetical classically 
1 a, Š s 
lis ioned response and an instrumental response that we have been 
discussing, 
In » AYA . è 
the avoidance-learning experiment cl 


or less 4 i r š S 5s 
s on the base line because the CS is associated with shock on trials on 
not occur. But fear conditioning can 


hen we seek to separate the Pav- 
ag. This separation can 
as the “acquired-drive 


assical conditioning occurs more 


whi pa 
A ich the avoidance response does 
also arr . S ws 
ae be accomplished off the base line, w 
Viz sa ; 
an aspect and the instrumental aspect of learnir 


be . 
"i achieved by means of what has become known 
b4 ri m R s, 
periment.” Miller (1948) put rats In à white compartment, shocked them 
a door into an adjoining black 


sn anid required them to run through 0 bia 
7 partment to escape the shock. After a series of such escape-training 
trials, Miller argued, fear should be elicited by 
the “Sues arising from the white shock compartmen 
havior had been established and fear had supposedly been conditioned, the 
door between the two compartments was closed, and the animals were 
required to learn a new response: manually turning a small wheel to open 
the door to the adjoining black compartment. Many of Miller’s animals 
showed no learning; their reaction to fear was to freeze or to crouch in the 
white compartment. But about half the group did learn the required 
response, Miller argued that the acquisition of the new response was Te- 


Inforced by the fear reduction that it made possible. Then he argued further 
that both the new wheel-turning response and the inappropriate response 
(freezing) were motivated by fear elicited by cues in the white compartment. 
This situation is thus somewh edure but essentially the 


at different in proc 
same in concept as the conventional avoidance-learning experiment. The 
chief difference is that fear is learned 


1 off the base line before the animal is 
put on the base-line task. 


; Another ingenious approach 
line was reported by Solomon and 
trained dogs on an avoidance task; t 
the dogs were subjected to a series of Pavlovi 
paired with shock. This experience was given 0 Give 
dogs were deeply curarized. (The purpose of the curarization was to prevent 
any effect on the instrumental base-line behavior from being mediated by 


some inadvertently learned skeletal response during the Pavlovian session.) 


apparatus cues, specifically 
t4 After the escape be- 


Pavlovian fear off the base 
] Turner (1962). These investigators first 
he training was then interrupted and 
an trials in which a tone was 
ff the base line while the 


to establishing 


d Mowrer in assuming that this fear is classically condi- 
d earlier, did not subscribe to this view. 


a Most theorists have followe 
tioned, but at that time Miller, as we have note 
iment can be interpreted as another instance of 


Nonetheless, the results of his experi 
Pavlovian-instrumental interaction. 
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Later, when the effects 
the base-line avoidanc 
stimulus that had been 
just 


of the drug had worn off, the dogs were laren 
e task and the customary CS was replaced by ed 
a cue for shock in the Pavlovian session. It produc 

as much and perhaps a bit more 
with which the animals h 
while the dogs were cur 
tone and that this new 
learned av 


avoidance performance than the eA 
ad been trained. The interpretation was wr 
arized, fear had been classically conditioned to br 
Source of fear then served to motivate the we 
oidance response. 


> Je eke 
; AET ARS ; Turner's © 
There was another important innovation in Solomon and Ti 
periment. During the P 


stimulus associated with 
addition 
were 


avlovian session, the dogs not only reah 
shock; they also received from time to Rei 
al stimulus that was not associated with shock. The experimen 
attempting to bring the fear 
that it would be elicited by the 
the animals were returned to th 
was introduced in place of the C 
performance: either no respondi 
lovian principles, the neg DEET 
inhibitor, It should have ar might be already a ne 
for example, fear elicited by apparatus cues. This general view O 
excitation and inhibition of 


a Pavlovian fear re 
and considerably strengthened by 


and LoLordo (1965). The b 
Turner's experi 


reaction under discriminative control a 
one stimulus and not by the other. W a 
e base-line task and the negative ina 
S, the result was a decrement in ee A 
ng or rapid extinction. According to) ed 
ative stimulus should have become a condition 

inhibited whatever fez 


action was greatly extended 
a series of studies reported by a 
asic concept was the same as in Solomon @ 4 
ment. Dogs were first trained on an avoidance task and A 
persed sessions in which the dogs were not curarized but we! 
confined to one p 


een a 
art of the shuttle-box @pparatus and there subjected to 


3 : i shock 
series of tones and shocks arranged so that one tone always predicted sho 
and the other tone always predicted 


a period of time free of shock, Rescorla 
and LoLordo found that when the S+ for shock Was introduced as a probe 
on the avoidance base line it Produced a marked ac celeration of avoidance 
behavior, and that when the S— for shock was 
produced a marked inhibition ofa 


k it 
introduced as a probe i 
; voidance. One of the elegant features O 
this study was that Rescorla 
hibition procedures such a 


a E w M in- 

ee variety of Pavlovian re 
s condi ; ops i- 

; ishi 9 on and differential inhib 

tion for establishing the § 

producing inhibition led to 


ew: v ž : r 
avlovian techniques $ 
a -o ausits 
ance on the base-line avoi 
ance task. Rescorla and LoLordo thus provi 
argument that avoid 


ded stron support for the 
ance behavior in their și 8 PI 
classically conditione 


i Seen Seems to obey many of the general 
principles of Pavlovian conditioning. The argumen, for 4 conditioning 
interpretation of the secondary learning System i 

at this point. 


lookeq rather convincing 


The Logical and Empirical Status of re 157 


The Logical and Empirical Status of i 


identify, and objectify the 


We , 
have seen that the attempts to localize, 
situation any better 


appetitiv 
k Aponte se pies ost emai 1s; the i ; 
with the conditioni ear neactions There is an enormous literature dealing 
sin eee ioning of heart rate and other reactions which in some com- 
Gionneead commonly assumed to constitute fear. Again, 1t would be an 
the one an ae the interpretation of instrumental behavior and in 
Seri aes pr of its motivation if we could apply what 1s known about 
the: fear E poems for example, to the problem. if we could monitor 
theavords oe is assumed to motivate avoidance behavior independently of 
and provi te behavior itself, we could verify the entire theoretical structure 
learning rh direct confirmation of the theoretical ps ian 7a eon 
cing smem Una apr ee et 
iite ines a here is no more comp. i $ pa Daten poe Po 
Sorre bba piat which deals with the classical conditioning of hear jate; 
Resin 1e difficulty can be attributed to the complexity of the echap. 
tae M mg the circulatory system. The main function of increased heart 
sorte ypsther it occurs in response to the US or to the CS, is to make possible 
violent defensive reaction, like fighting Or running away, by increasing 
the blood supply to the muscles that are involved in violent action. But the 
load Supply to the muscles can be increased in a variety of ways: by 
Speeding up the heart (increased heart rate), Dy increasing the stroke (the 
amount of blood pumped out in any one heart contraction), and by con- 
Stricting the blood vessels in the core of the body and dilating the blood 


Vess : ‘ aceite 45 iner i 
vessels in the periphery. At the same time, blood pressure 1s increased by 
any overall constriction of ssels. All these responses are neurally 


controlled and should therefore ionable. The problem is that any 
one change in the circulatory system is likely to trigger compensatory 
changes in the other mechanisms that tend to restore the overall balance. 
For example, if for some reason heart rate is suddenly increased, compensa- 
tory mechanisms will be stimulated to lower blood pressure and reduce the 


stroke, both of which tend to reduce heart rate. Thus, in a conditioning 
f the tot y system may be 


e response O ; 
affected in one direction compensatory mechanisms will tend to counteract 
cessary to monitor 


it. There is one clear methodological implication: It is ne 
a number of different cardiac responses such as pressure, stroke, and rate to 


discover how the total system is responding in any given situation. If we 


fail to find heart-rate conditioning in 4 Pavlovian experiment, it may simply 
be because stroke or pressure is the response that is being changed, and this 


change may be accompanied by an increase, a compensatory decrease, or no 


arterial ve 
be condit 


experiment, although on al circulator 
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5 — 5 ae species 
change at all in heart rate. A further complication is that different I 


i i i i ani 2 sen within a 
of animals have quite different circulatory mechanisms and even 


: Erg rah 4 : -ways in which the 
species there may be great individual differences in the w ays in w D pe 
+ : a ast > ive yhe 
circulatory system responds during a Pavlovian experiment. Even w 

think we know all 


t n eve re may be 
about the circulatory system of our animal, we may 
surprised to find that conditioned re 


US, the CS, and the CS-US relation 

Students of heart-rate conditioni 
although heart rate and other index 
conditionable, there does not 
combination of responses tha 
mean that which motivates 


actions can change as a function of the 
shi >. m 

ng - been forced to conclude ns 
es of autonomic functioning are ee 
appear to be any one response or any Sar 
t can be equated with fear—if by “fear e 
avoidance behavior and suppresses PPA OET 
behavior. In their review of the problem Rescorla and Solomon (I* 
concluded, as they did in the case of rg, th 
fear response, let us call it rp, 
fiction because a numbe 


at the hypothetical nag ae 
appears to be a convenient fiction. It a ie 
r of very capable researchers have failed to fine i 
and convenient because it seems to have many of the functional prope? a 
shown by classically conditioned responses. But again Rescorla and Soler “lis 
have concluded that it is the procedure that matters, It is clear that er 
lishing a relationship between a cue and a consequence, such as a tone aoi 
shock or a tone and the absence of shock, in a Pavlovian manner ne 
the cue to have a dramatic effect upon instrumental behavior. It is ane 
according to Rescorla and Solomon, that the Pavlovian procedure is effect! 
whether or not any particular observable response is conditioned. 
Perhaps it is not very important whether rẹ and rp are identifiable 
responses. Certainly most of the time the hypothetical fear reaction, a 
like the incentive motivator ro, is treated as a hypothetical construct an 
inferred from its effects on behavior. If these mediating reactions should 
ultimately prove to be central events, occurring in the central nervous sys- 


tem, then they may still be hypothesized to have the functional properties 
of classically conditioned changes in heart rate and s 


alivation. But this 
argument is no good. The fact is that th al properties are no 
longer very clear—that is why th urthermore, the whole 
ndary learning system 
a as . the Pavlovian inter- 
pretation was that the mediating mo 


Te and ry are merely 


There is th 
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Havi 
ing come i 
to the brink of concluding that salivation is not a classically 
consider how classical 


conditi 

oned res TPE 

conien ra it is clear that we must stop to 

rs 5 

g should be conceptualized. It isa good thing to conclude with 


Res 
corla an 
and Sol ai : 
omon that it is the Pavlovian procedures that are important 
ent instrumental behavior. 


bec 
ause they hav 
Butit ent ar dramatic effects upon subsequ 
uC 0 k . : . $ + 
be a far better thing to know why. Classical conditioning has 


to be 
more than jus 
than just a procedure—but what more? 


Classical Conditioning 


arning necessarily consists of 
trengthening an already existing 
the moment, we may ask: 
in the Pavlovian situation? 
plica of the UR. The 


Interpretations of 

It is 
A Mioa taken for granted that le 
S-R a to a stimulus or of s 
What is the ies Granting this assumption for 
We have E of the learned response (CR) 
animal does cady seen that the CR may not be a Te l 
US. Hanko noe necessarily react in the same way to the CS as it does to the 
gested vette wiih this apparent anomaly, a number of writers have sug- 
re CR serves a preparatory function. It prepares the animal to 
S. In some instances, as in eye-blink 


cope i 

ns y : X 
some way with the upcoming US 

i , means of puffs of air to the eye 


condition; 
ee which is usually established by l ; e 
and it ci JR looks very much like th eye k to the air put, 
whan ne be argued that closing the eye defends it in e way 
ten s See threatened air puff as agains : puff. In S a oe 
evden CR should resemble the UR, am p are or ya m 
Sivas i membrane, the third eyelid that some als have, evidently 
enous HLIONS: other than protecting the eye because it does not close 
the On to give the eye much protection: but in spite of this curious feature 
Wons ‘ of the nictitating membrane again resembles the UR. We may 
Tita r: If this CR is not a preparatory response, is ìt a legitimate replica? 
lesen case of stimulus substitution of the sort that was suggested by earlier 
HADE theorists as the model for all classical conditioning? 
Salivation can also be said to be a preparatory response. The usual US 
-< q relatively hard-to-consume dry 


ae salivary-conditioning experim : : x 
powder. By salivating the anima r able to ingest and digest 1t. 
So, although the CR looks like the UR ir we cannot be sure 
whether it is a replica of the UR or a preparatory response. The fact that 
salivation can also be conditioned with dilute acid as the US does not 


clarify the issue. 
ee : another curious conditi 
eae the complexity of the bod 
Ae TE of whether the 
SN ang Hutton (1969) reported th 

(insulin drives down the blood 


phenomenon that suggests some- 
ys adjustment mechanisms and again 
CR is preparatory in nature. Woods, 
at when an insulin injection is used 
glucose level) and any one of a 


oning 


160 The Status Quo 


number of salient stimuli is used 


s ? aa oide 
as the CS, it is possible to obtain a cc 
tioned blood-sugar reaction. 


After a number of CS-US pairings, the CS ey 
produces a replica of the insulin reaction, a decrease in blood glucose sete 
But another investigator, Siegel (1972), who has also found singe nat 
using very similar experimental procedures, reports that the CR polis a 
of a rise in blood glucose level, Siegel suggests that the rise in blood ho 
isa preparatory response, occurring in anticipation of the effect of Gaines 
Whatever the underlying mechanisms producing the difference pean 
Woods’ and Siegel's results, it can be concluded that the CR may i 
the UR, it may be a preparatory reaction of the same or somewhat diffe 
form, or it may even be something altogether different. 


PEP i é ] ‘sts that what is 
The unpredictability of the CR relative to the UR suggests that 
learned in a Pavlovian situ 


ing 
ation is not a response at all. Perhaps pessoa 
else, perhaps something about the $-S* (CS-US) relationship itself, is pcan 
The point was nicely made in an early paper by Zener (1937). Zener ae “Ay 
Pavlovian salivary conditioning in a dog, but in contrast to Pavlovs Bu pet 
controlled experiments Zener let his animal move around freely a aig 
experimental space, and he observed all of its behavior, Zener found salivi 

conditioning, but he also found the dog displ 
behavior such as moving around 


ing a great deal of G 

restlessly during the intertrial somo 
going over to the food-delivery mechanism when the CS came on, Wisp e 
its tail, and barking at the CS. In short, Z 
tion of anticipating or expecting food, fad 

Perhaps, then, what is learned in a Pavlovian experiment is not a a 
ticular response to the CS but an expectancy of the US. If the experimente? 
only observes salivation or if he takes the trouble Pavlov did to restrain oe 
animal and to train it to hold still, then he is only going to sce the salivary 
response. But in the unrestrained dog it is clear that what is learned az 
something much more cognitive, Something more like a perceptual recogni- 
tion of the relationship between § and S*, 

So we see that one of asi assumptions ‘ 
conditioning Process is open to question. Even more remarkable is that in 
the last few years serious questions about the basic nature of the conditioning 
procedure have been raised. Nothing is sacred, it seems, in the world of the 

time of Pavloy it has been assumed that, what- 


learning theorist. From the 
er conditioning is, i “o ? : 
z ERAT S S and US are closely contiguous. 
essentia r - : a 
sends Pis A mng procedure is the pairing of 
CS and US. But let us consider iea eh ig 
‘ : we ena that indicate that pairing 
per se has little to do with co. l Y 
2 gear re 1 these Phenomena have a 
tracted a good deal of interest 


s sery indica- 
ener’s dog showed every in 


1¢ 
about the nature of tl 


in r ” 
; wee ok n recent years because both promise to alter 
drastically the prevailing view of what Pavlovian Conditioning is | ll about. 
Rescorla (1968) trained rats to Press a bay ag is all a 


for food; then he gave them 
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fear wae . 
task S oi B and mer returned them to the base-line 
KA OA oe Pe upp ess E me responding. Rescorla s study 
describing PRS aie gro the main point can be illustrated by 
Pavlovian ssc s ae sie iffered in the treatment received during the 
Bec SE av desl ne gale Each group received a series of CSs and 
atthe eia rs ` scatter ed randomly through the CSs but arranged so 
me ve > ability of shock in any given CS was 40. Each group thus re- 
differed ne mean number of CSs, USS, and CS-US pairings; the groups 
nons ve in ii events that were scheduled to occur between CS presen- 
Was “les ‘a ae group the probability of shock in the absence of the cs 
so 40, for another group this probability was reduced to .20, and for 
10. To summarize, during the Pavlovian, off- 
at the three groups received the same 
of shocks during the CS intervals but 
e non-CS intervals. When tested 
roduce severe suppression in the 
and virtually no sup- 
f fear had been 
40-.10 but that 
CS in group 40-.40. In spite 
mber of CS-US pairings, in spite of 
al shocks and that a great deal 
otal experimental situation, 


he snd opts o 
fuehey oe pew gre epe iaa 
differen ane the same probability c 
Sne i ia obabilities of shock during th 
40-10 seline task, the CS was found to p 
AN A less suppression 1n the .40-.20 group, 
toads n the A040 group. It appear at a good deal o 
PRD to the CS by the Pavlovian tre? 
of hive ) no fear had been conditioned to the 
e fact that there were the same nur 
oe fact that group 40-40 received more tot 
Of fear must have been conditioned to the t 


t i á H Ras ~g 
here was evidently no fear conditioned to the CS. 

becomes conditioned to 
e it in simple language, when the CS predicts 


ed shock because it was four times 
han in its absence, but for group 
fear was conditioned to it. 


s th 
tment in group 


a CS not when it is 


It appears, then, that fear 
Paired with shock but, to stat 
shock. For group .10-.10 the CS predict 
more likely in the presence of the CS t 
40-40 the CS did not predict shock, and so no 
To state the point in Rescorla’s own more formal language, fear condition- 
a “contingency” between CS and US. “Contin- 
s of relative probabilities. If the probability 
of the CS than in the absence of the CS, 
and this fear can then be manifest in the 
facilitation of avoidance behavior. 
1 the US and not the contiguous 
not the Pavlovian pro- 


i nia 

ng requires that there be 
4 peg s 
gency" is defined here in term 
of shock is greater in the presence 


then fear will be conditioned to it, 
titive behavior Or the 
een the CS and 
whether or 


suppression of appe 
It is this contingency betw 
pairing of stimulus events that governs 
cedure will be effective. 

: Recall the experiment by 
in which one tone had been exp 
Pavlovian sessions an S— OY safety 
that, although a number of shocks occurred, 


a and LoLordo (1965), described earlier, 
unpaired with shock. During the 
been established by ensuring 
none of them followed the 


Rescorl 
licitly 
signal had 
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tone. Recall that when this S— 


behavior, it produced a decremen 
had come to act as a Pavlov 
Rescorla a 


was then probed on base-line ein 
t in performance, indicating that ne ne 
ian conditioned inhibitor. In this T 
rgues, the contingency relationship is simply reversed; in ne 
lishing the safety signal the Pavlovian contingencies were arranged at aie 
the probability of shock in the presence of the CS was less than u aed 
absence of the CS. Under these conditions the CS becomes a conditio 
inhibitor, To put it in more simple language, if 
Predicts no shock (in a situation in which 
stimulus wil] become an in 
wise be present, 


There is one further implication of Rescorla’s 
relationship. What would we predict if the 
same in both the presence and the 
neutral, an So, neither a Payloy 
is precisely the Way in w 
Sroup .40-.40 in the 
Prediction Se h 


stimulus periei 
tha 
shocks do occur), then oe 
ill inhibi f i ot 
hibitor and will inhibit fear that might 


analysis of the cpnungen 
probability of shock were As 
absence of the CS? The CS should a 
ian excitor nor a Pavlovian inhibitor. Tha 
hich the experiment 


Experiment just describ 
ad no e 


al conditions were arranged for 
ed, and in accordance yri ve 
ffect on base-line performance. Rescorla (1972) ha 
of experiments (mostly fr 
vide striking Support for his new interpret 
be in very close 
fear seems to be condi 
inhibition of fear to cues th 
much at all to cues that be 
as in Rescorla’s 196 


om his own research) that Pee 
ation of Pavlovian conditioning: 
agreement with his contingency theory- 
tioned to cues t 


ji “k, the 
hat predict shock, t 
at predict freedom fr 


om shock, and nothing 
ation to shock, even when 
uently paired with shock. is 
Sults from the pairing of CS and l 7 

as run into trouble in the Phenomena of overshadowing 
and blocking, Pavlov (1927) noted that, if conditioning were carried out 
a dim light and a loud noise, the 

irely upon the Stronger component of the CS. When 
it no CR, whereas the noise would 
ith which the dog 


ershadow” the weaker 
e latter 


mulus would “oy, 
one, so that no condition 


paired with food and even tho 
from the outset of conditio 
originally discovered in conne 
extensively studied by Kami 
tion, Kamin (1969) has disco 


8 procedures, In addi- 
calls blocking. In a typical 


Phenomenon, which he 
t a light AS: paia ith a 
shock eight times (sufficient to produce a st 8 Paired w 


rong fear de aten ted. 
i E on, as indicate 
by suppression of bar pressing for food wł as ind 


reacti 
nen t — 
he light iS introduced as a 


ligt 
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probe). Th 
: e s ? A 
n for one group the light and tone are paired an additional 


eight times wi 
A aa So agin oa T when the light is introduced asa probe on the 
DhE no ae Sopp ry but when the tone is introduced as a 
Sf ins ides teehee seit xt gund A control group that first receives pairings 
shows aprecio . s 1ock and then pairings of just the tone with shock 
line. The Sie 5 when either the light or the tone is probed on the base 
ese ihe light : rom the control group indicate that the prior conditioning 
CS: butike ahi hot prevent the tone from also becoming an effective 
are EENEI a = other group show that when the light and the tone 
candies i ry. conditioning to the tone is — blocked” by the prior 
al tne nie 3 ie ight. (Of course, in all such studies the roles of the light 
ttet nT interchanged to balance out any intrinsic differences among the 
) 


The = 
ty ç phenomena of overshadowing aà 
pairing a stimulus with shock is not sufficient to condition fear to it and 


Suggest P p X aai a 
st a number of new ways of looking at conditioning. One way is to 


nd blocking indicate again that sim- 


importance of the original response to 
enting or investigatory reflex to the CS 
ying that the 


had emphasized the 


must be some ori 


5 Let us recall tl 
s this is just a behavioristic way of sa 


Hed ws the learning of the predictive relationship between S and S*. 
by Pai Gi information about s*, or, to use the phraseology first introduced 
nares ov, the CS “signals” the US. Consider the following interesting phe- 
Blasi on also reported by Kamin (1969). If in the critical phase of the 

>cking experiment, that is, at the point where the light and tone are both 
paired with the shock, a new S* is introduced, or if it is altered (for example, 
increased in intensity), then blocking does not occur. A recent report by 
Gray and Appignanesi (1973) suggests that simply accompanying the shock 
with a relatively innocuous stimulus, one that itself would not support 
conditioning, also relieves the blocking effect. In other words, it looks as 
though, if the light plus the tone predicts an s* that is different in almost 
any way from that predicted by the light alone, then conditioning to the 
tone will occur. If the light already predicts shock, then the tone is re- 
dundant. It provides no new information about what is going to happen, 
and perhaps for that reason no conditioning occurs. Thus, one possible 
new interpretation of conditioning is that it is a procedure that permits 
the animal to process new information about environmental events. 

An alternative view of conditioning, which has an even stronger cognitive 
flavor, attributes blocking to hypothetical attentional processes of the ani- 
mal, When the light has been paired with shock a number of times, the 
animal’s attention is directed entirely to the light and the tone is not per- 
ceived. Since it is Net perceived, no conditioning to it can occur.® Still an- 

hat Pavlov 
the CS. He suggested that there ? 
if conditioning is to cco Perhap: 
animal must pay attention tO the CS. 
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other approach to the phenomenon of bl 
and Wagner (1972). They 
from which c; 


ocking is that suggested by Lenco 
propose a semiquantitative model of tere A 
an be derived many of the facts of overshadowing, as wel al i 
variety of the contingency effects previously reported by Rescorla. ors z 
their model defines y as the strength of conditioning, and al a as ns 
crease in strength of conditioning to stimulus a on a given trial. ‘Their mon 
States that this increment, AV, =C (A — Va). Here G is a constant ie 
depends upon the specific properties of the CS and the US, A is thepess fie 
Strength of conditioning obtainable with the given US, and Lo iny 
strength of conditioning that already exists for stimulus a together we -i 
other stimuli X that may be present, This equation combined with p of 
equations for other stimuli present in the Situation carries a num E Ave 
implications. One is that conditioning to a particular stimulus, such hee 
tone in the blocking experiment, will be minimal if there is already a sa 
conditioned reaction to other stimuli in the 
Rescorla and Wagner's model is an 
describing the results of many kinds of 
larly those in which there is more th 
ground stimuli are 
to be in fear-conditi 
It does not tell us a 
a matter of each man for himself whe 


situation such as the can 
ingenious and powerful ee 
conditioning experiments, oes 
an one CS and those in which are 
an important part of the total situation (as they oe 
oning experiments). But the model is purely EeP 
nything about the underlying learning process. It is ee 
n it comes to explaining what gat 
ina Pavlovian experiment, Perhaps the Pavlovian procedure miply ae 
an animal to learn an expectancy for shock; much of the recent data 1S 


< ~ m S g n 
consistent with such an interpretation, Perhaps conditioning is based upo 
some kind of reinforcement mechanism; however, 


ing process is itself in considerable doubt at the 
shortly, Perhaps some new contiguity theory wil] 
restore order and to give us a new coherent pict 
process. But as it stands now, it is fay from clear ho 
the classical-conditioning paradigm, 

basic paradigm itself should 


reinforcement as a learn- 
moment, as we shall see 
be put forward soon ie 
ure of the conditioning 
W learning occurs within 


and it is even far from clear how the 
be conceptualized, 


Thorndike painted a simple Picture in which 
responding have automatic strengthening effects on 
nection. Hull simply elaborated this idea 4 
underlying his primary-learning sys 
lated over the last twenty years h 


certain consequences of 


any preceding S-R con- 
ade it the b 
reat mass of 


asic postulate 


evidence accumu- 


ated that the primary- 
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learni 
arming syst se relativ = * s 
g system is relatively ineffective and provides a rather inadequate 


account 
of ‘fornia . 
the performance, much less the learning, that occurs even in 
een that there is very 


s ; 
a e E situations. We have s 
wiat KA EEY ev gi among those who have defended Hull's system, on 
ent consists of; perhaps 1t 1s drive reduction, perhaps drive 
s controlled by contingent reinforcement, it 
upon the secondary-learning system, upon 
and secondary reinforcers. Mowrer 
imary-learning 


with the entire pri 
attributed the strengthening of 
t is assumed to 


induction, Even if behavior i 
nes also depend to a great extent 
(1960) «ai er learned fears, 
astern aa 4 became so disenchanted 
at he abandoned it completely and 
se of incentive motivation tha 
ogressively strength- 


It is not that food pr 
li associated with food increasingly 


ithin the reinforce- 


Dbehavi 5 
o havior entirely to the incr 
ceur wlien reai r 

r when reinforcement is given. 
it is that stimu 


The learning that 
not the building of an S-R connection but the 


ly conditioned motivators, mediating responses that 
' associatively and motivationally. In short, 
1 of thought appears to have little faith 
a description of the learning process. 
_ Psychologists of the Skinnerian persuasion continue to emphasize the 
importance of reinforcement in the control of behavior, but they too have 
drifted away from reinfo a learning mechanism. Initially (Skinner, 
1938) it was believed tha produces learning. But in the last 
twenty years or so it has become parad atic thayreinforcement is nothing 
more than a procedure for controlling behavior, and the question of learn- 

ed in controlling 


ing is left in abeyance.) Those who are primarily interest 
behavior and making it highly predictable, particularly if they are interested 
in the predictability of asymptouc behavior, must almost necessarily ignore 
arning. Learning ™ 1 changes, instability, 


eans transitiona 
edictability- If the Skinnerian runs enough sessions, 
ar—along W 


ens the . 
` as the running habit: 
motiva pi s 
ii ivate approach behavior. 

ent paradigm is, therefore, 


occurs W: 


establishment of classical 


contr : 
ntrol the instrumental response 
lian schoo 


radigm as 


wha e $ 
; hat remains of the Hul 
in ae 

the reinforcement pa 


rcement as 
t reinforcement 
igmi 


ith the learning data. 

operant conditioners over 
al principles that look like 
that reinforcement is more 
Such a principle looks as if 
ment mechanism. But in 


the question of le 


and momentary UNpr 
all these problems disappe 
Apart from these jdeological confines, many 

e indicated a faith in certain practic 
has been noted, for example, 
quickly after the response. 
ief in a reinforce 

derlying learning process 


the years hav 
laws of learning. It 
effective if it follows 

it is derived from an impl 
the last few years even this implic of | 
has been lost, and even the practical princip 
fall by the wayside. Herrnstein (1969) proposed that in the case of avoid- 
vior the event that controls it (and at this point it hardly makes 
a reinforcement contingency) is the reduction in the rate of 


ance beha 
roduces. Herrnstein and Hineline (1966) found that 


sense to call it 
shocks the behavior P 
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i iti i i aintained 
under appropriate conditions avoidance behavior could be ma 


when responding simply decreased the probability of randomly programm 
shocks. There is no localized or identifiable consequence of the EER 
nothing like Mowrer’s fear reduction or even CS termination that n 
the behavior; it is simply maintained by the reduction in the number 
shocks per minute, 

Herrnstein (1970) has extended the 


: l appeti- 
general argument to the case of app 
tive behavior. He 


k x uatan in whieh 
proposes, for example, that in a choice situation in w 


; kí stimuli to 
an animal has two responses that can be made or two kinds of stii 
respond to, behavior is distributed i 


: Š : e A ae any food 
For example, if the Tat in a choice situation receives twice as many 


s as it does to another, it will Fenech 
twice as often to that stimulus. Again, it hardly seems appropriate 1O: TE 3 
to the food pellet as a reinforcer because the discrete event of regde 2 
food pellet in itself is no longer related to the behavior; it is the oe 
frequency with which food is received Over a period of time with which ! E 
animal’s behavior is correlated. Baum (1973) has put the argument in on 
stronger language, He says that contiguity between the response and t 7 
reinforcer is immaterial: it is the correlation between behavior and its Pig 
sequences over time that matters, According to Baum, reinforcement 3 ; 
contingent program of events, rather than a discrete event.® Our MEWS 9 
the basic operant paradigm are changing. It is still true that the skillful use 
of contingent reinforcement is the best technique we have for controlling 
Operant behavior: so the old paradigm retains considerable merit, particu- 
larly in the practical world of everyday affairs. But we need to understand 


why it works, when it works, and why it so often fails to work. In Chapter 2 
we focus upon this troublesome matter, 


In summary, it is clear that many learnir 
elements that characterize both of the b 
typical operant Situation provides the o both reinforcement 
learning and classical conditioning to occur, It is widely assumed that both 
kinds of learning do occur and that they are reflected in different aspects 
of behavior. ee Tainig is assumed to determine what the 
response is, and classical con 1uoning is assume termi > strongl 
modai it is. Although the two basic pirat determine how strongly 
tinctly different procedures, there are di ne 
critical features of these procedure 
what the underlying learning pro 


1g situations contain procedural 


asic paradigms. For example, the 
Pportunity for 


iets about just what the 
S are, y 


E ips: €n less consensus about 
cess is in either case 
6 This view of the reinforcement 


relationshi 
Rescorla's (1968) view of classical condi 


P bears an 


tioning. In bow interesting resemblance to 

f SORN Cases learning is considered to be 
based upon the patterning of Events over time rather than “PON discrete event isolated 
in time. In both cases the learning appears to Sonsist of an a ee 


i Teciati x; 
rather than an automatic strengthening of a response, PP ation of the pattern 
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rh ianessst of thinking about two-fa 
Trapold a pa oF Mowie (1947), Logan ( 
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and complicated literature. The interested reader might well st 
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CHAPTER 
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THE STRUCTURE 
OF LEARNING 


Whatever differences there ma 
have shared a common optimisti 
versality of nature’s laws: 


y have been among them, all our theorists 
c attitude tow 


ard the uniformity and uni- 
the attitude that i 


f we can only once discover 


© doubt this faith in the uni- 
formity of nature and in the universality of th ing that justified 
to Pavlov his building a general theory of learning by studying only one 
response system in one species of animal. What Pavlov discovered about 
salivation in the dog was supposed to reveal principles that would be uni- 
formly applicable to all learning. This optimistic faith in the uniformity of 
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nature was by no means unique to Pavloy; it is shared by virtually all learn- 
ing theorists. Consider the following quotation: 


In studying (operant) behavior we must make certain preliminary deci- 
sions. We begin by choosing an organism—one which we hope will be 
representative but which is first merely convenient. We must also 


choose a bit of behavior—not for any intrinsic or dramatic interest it 
may have, but because it is easily observed . . . can be readily recorded, 
and . . . may be repeated many times without fatigue. Thirdly we 
must select or construct an experimental space which can be well con- 
trolled. (Skinner, 1957b, p. 344) 


This quotation indicates that at that time Skinner had a profound faith 
in the universality of the principles controlling behavior, But what is this 
strange concept of a “representative” animal? Whom does the pigeon 
represent? Does it stand for you or me, the rat, the flatworm, or even other 
birds? What is this strange notion that we can come to a better understand- 
ing of animal behavior by looking at a response which is of no importance 
but is easily observed and can occur at a high rate? And what is this strange 
obligation to seek an understanding of behavior by looking at it only in 
situations such as the Skinner box? j 

The present chapter will show that the assumption of uniformity can no 
longer be defended and that our principles of learning no longer have a 
claim to universality. We will see that learning depends in very important 
ways upon the kind of animal that is being considered, the kind of behavior 
that is required of it, and the kind of situation in which the behavior occurs. 
Learning has a structure that must be discerned through an analysis of 
cases where learning occurs very rapidly and other cases where learning 
occurs either slowly or not at all.1 Let us consider first an instance in which 
learning occurs with great rapidity under seemingly impossible conditions. 


The Garcia Effect 


Many psychologists were awakened to the fact that learning is much 
more likely to occur under some conditions than under others by the re- 
markable work of John Garcia. In one of the first and perhaps the most 
remarkable of these experiments, Garcia and Koelling (1966) trained rats 
to drink bright, noisy water. This feat was accomplished by attaching an 


1 Recent writers, perhaps struck mostly by some dramatic failures of learning, have 
Popularized the phrase “constraints on learning” in referring to the nonuniformity of 
the laws of learning. But this phrase tells only half the story, for there are also “facilita- 
tions of learning.” It is not that learning is limited; it is the applicability of our present 
laws of learning, and perhaps the laws themselves, that are limited, k 
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electrode to the spout of the drinking tube so that whenever the tat s E 
touched it the appropriate electrical circuitry produced flashing ligit a in 
a raucous noise. For one group of animals drinking bright, noisy wate Bis 
followed shortly by electric shock to the feet: Drinking bright, n pe 
was punished by shock. When offered bright, noisy water the NEAL day, en e 
animals showed an aversion to it and would drink little of it. A cour 
group of animals also drank bright, noisy water, but the ee eed 
doing so was different. This group was made ill (either by being X-irra ei 
or by being injected with a toxic substance), When this group was 3 i 
bright, noisy water again, they showed no aversion to it and drank as mt 

as they had initially. A third group of animals was : inedi 
the lights and noises but with the distinctive taste of saccharin, and imn E 
ately following the consumption of the saccharin solution these one 
were given foot shock, In contrast to the first group, however, they AER 
no evidence of a learned aversion on the test day; their drinking of the a f 
water was not reduced. The fourth group was also given the novel taste Z 
saccharin water and were then made ill. When these animals were tested th 


p 4 ean i i These 
following day, they showed a strong aversion to the saccharin solution. 
results are summarized in Figure 9.1. 


offered water without 


Consequences 


taste 
Cues 


audio-visual 


Fig. 9.1 Results of the bright 
ment showing how 


» noisy water experi- 
h learning was found 
only wi inati 


of cues and conse- 


& Koelling, 
Sciences, 1966, 4, 123-124.) 


Psychonomic 


Notice how elegantly the study was counterbalanced, It cannot be argued, 
for example, that one group failed to learn an aversion because illness is añ 
ineffective consequence, for another group did learn with this consequence. 
Nor can it be argued that foot shock is not an effective Punisher. because St 
was effective for another group i 4 


; - Similarly, it cannot be argued that one 
group failed to learn because bright, noisy water is a poor cue, for it was a 


good cue for another group. Every element was counterbalanced. How are 


we to explain the asymmetrical results Produceq by the four groups? Garcia 
and Koelling’s interpretation was Simply that some things ie more learn: 


able than others. Whether or not learning Occurred in z particular group 
z lationshi betw cua 
was a function of the re P veen the cue pr F 
perties of the water 
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and the consequence. Indeed, the title of their paper was “Relation of Cue 
to Consequence in Avoidance Learning.” If the relationship is suitable, for 
example, if both cue and consequence are external events such as light and 
sound followed by shock, then learning is readily obtained. If the relation- 
ship involves only the digestive system—the taste of water followed by ill- 
ness—again, learning is readily found. But the attempt to train an associa- 
tion that runs across these two systems was not productive. In short, Garcia 
and Koelling showed that the effectiveness of a learning procedure depends 
dramatically upon there being some intrinsic relationship between the cue 
and the consequence. The remarkable facility with which rats (and a num- 
) learn about the relationship between the taste of a 
and a subsequent illness we shall call the “Garcia 


ber of other animals 
particular food substance 
effect.”? 

There are some remarkable features of food-aversion learning. One is 
that, in contrast to the learning found in most situations, it occurs with 
remarkable rapidity. Three or four pairings of a cue and consequence are 
sometimes used, but the effect can be readily shown after a single pairing. 
Another unusual feature of the Garcia effect is that it is possible to introduce 
a delay of many hours between the consumption of food and the subsequent 
illness and still produce an aversion to the food. Shorter delays generally 
lead to greater aversions, but aversions have been shown with delays as long 
as eight hours. Unpublished results that are similar to delay gradients 
reported by other investigators are shown in Figure 9.2. Thus, food-aversion 
learning occurs under conditions that cannot produce learning, according 
to traditional theories; there are not enough trials and the delays are much 
too long, Punishment can be effective after a single trial but not when it is 
delayed more than a few seconds. What makes food-aversion learning possi- 
ble is that the rat has a peculiar facility for just this kind of learning. 
Indeed, it can be argued that such learning is essential for an animal that 
occupies the ecological niche in which the rat is so successful. The rat is an 
opportunistic forager and a generalized eater, and if it is to survive eating 
a great variety of foods it must have a learning mechanism to protect it 
are poisonous. We might almost expect the rat and other 


against those that 
ecological niches to have evolved the ability to learn 


animals in similar 
under such conditions. 


2 The fact that rats will learn not to eat food that has been associated with illness had 
already been demonstrated by Garcia in earlier studies of the aversive effects of X-radia- 
tion. It had been previously noted by Rzoska (1953) in his studies of rat eradication that. 
while the introduction of poisonous food might kill most of a rat colony, survivors of thie 
initial poisoning would refuse to eat the poisoned food again, But the phenomenon had 
not been considered from the point of view of learning theory, and it is because Garcia 
first saw the theoretical importance of the phenomenon that we can properly call it the 
Garcia effect. i 
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Hours of delay 
Fig. 92 The percentage of saccharin consumed in 
a choice test between water and saccharin as a func- 
tion of the delay between the previous drinking of 
saccharin and being made sick. Nonpoisoned control 
rats preferred saccharin 2 to 1 over water. (Unpub- 
lished data of A. L. Riley.) 


The Garcia effect has consider 
using a variety of toxi 
illness, and perhaps 
with a variety of test 
flavors a 
a pheno: 


able generality. It can be Jonona 
c agents: different kinds of poison, radia nan MAES 
any kind of bodily disturbance, It can be demonstrate 
foods, but there are some limitations. One is that some 
Ppear to be much more readily associated w 
menon that has been called ‘ 
are inherently more discriminable th 
however, that these differences are not due to the in 
stimulation but reflect the animal’s f 
Similar to it. A second limitation is that virtually 
the Garcia effect occurs much more re 

than with familiar ones. Indeed, w 
the effect is virtually impossible t 


ith illness than others, 
‘salience,” suggesting that some flavors 
an others, Kalat (1974) has determined, 
trinsic quality of the 
amiliarity with the test flavor or flavors 
all investigators find that 
adily in connection with novel flavors 
ith highly familiar substances like water 
o demonstrate, 
So far in our discussion of the Garci 


tion that learning either occurs or does 
was described 


a effect there h 


not; the bright, noisy water study 
as if that were so. But it shor 


ild be 
always a continuity of effects. The 
learned; it is really le 


as been an implica- 


ither learned or not 
‘arned to a greater or le Sree. Effects are always 
nt shown in Figure 9.2. 


and illness, the stronger the 
eater the illness 
er of pairin 
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al between ingestion 
also known that the gri 
the resulting aversion. The greater the numb 
the consequence, the greater the aversion. Even with cues 
or of low salience it is still possible to show aves. SEEGE 
intervals are short, the trials many, and the illness serious, an veneral, we 
have to think of the learning in quantitative ate a ae z 
demonstrated under a great variety of conditions, ay 
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learning occurs only under optimum conditions. It even seems likely that 
rats could learn to avoid bright, noisy water if doing so were followed by 
illness if training were extended, and other experimental parameters were 
optimized. There is no reason to suppose that the rat is incapable of such 
learning, but it is evidently not easy. The main point of the bright, noisy 
at is not that learning is impossible under some conditions 
facilitated if the situation is arranged to take 
al predisposition to learn the relationship be- 


water experimer 
but that it is enormously 
advantage of the rat's natur 


tween certain cues and certain consequences. 
This continuity of effects is nicely illustrated by recent work done at the 


University of Washington by Anthony Riley. Riley found that under cer- 
tain conditions rats will learn to avoid drinking in a particular place if 
such consumption is associated with illness. First, rats were trained to drink 
water from a bottle in a fixed location. Then the bottle was put in a new 
location, and drinking was followed by poisoning. If the animals were then 
given a one-bottle test, that is, if the water bottle was simply inserted in 
the new location on the day following illness, there was no reduction in the 
amount drunk. But given a two-bottle test, that is, the choice of drinking 
at either the new or the old location, the rats showed strong avoidance of 
the new location that had been associated with illness. This aversion ap- 
peared to be facilitated if the new place was “marked” with a new taste, 
like saccharin, but taste had to be held constant at the time of testing. If 
the animals were offered a choice between saccharin and water, they would 
regardless of where it was located. But, if both 
test bottles contained water, then strong aversion to the place associated with 
poison was found. It appears that the rat has a hierarchy of cue utilization. 
It will first avoid a taste if there has been a distinctive taste associated with 
But in the absence of a distinctive taste cue it will avoid a place 
that has been associated with poisoning. There is the further interesting 
complication that a distinctive taste helps to mark, set off, or call attention 
to a distinctive place, and by this means facilitates place learning. 
Similar studies by Garcia and others have shown that the rat does not 
readily learn to avoid foods that have been associated with distinctive odors 
paired with poisoning, 
dures like those in the place-learning experiment, Riley 


always avoid the saccharin, 


poisoning. 


but again this failure does not reflect all-or-none 


learning; using proce 
has been able to show 
Again certain methodological precautions have-to be taken; if there are 
taste differences, the rat will avoid the less familiar taste whether or not it 
associated with poisoning. But when taste is held constant an 
an be readily demonstrated. 


clear avoidance of odors associated with illness. 


has been 
aversion to odors ¢ 

In other animals the hierarchy of cue utilization may be quite different. 
Wilcoxon, Dragoin, and Kral (1971) gave blue, salty water to rats and to 
quail. After they had been made ill both species of animals were offered a 
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choice between blue water and sal 
water, which is consistent with their 
the quail avoided the blue water, 


sition that birds are visual eaters 
chickens w 


ty water. The rats avoided the =e 
r dependence upon taste cues, whee i 
which is consistent with the general propo 
- Earlier, Capretta (1961) had shown ee 
Ould not learn to avoid a taste that had been associated wit 


r p x 4 3 5 en ASSO- 
illness but would learn to avoid food of a given color that had be 
ciated with illness.3 


How are we to explain the Garcia effect? How are we to explain the 
remarkably rapid learning that occurs w S a appli 
kinds of consequences are used? Logically, both basic paradigms are apl ii 
cable. The Garcia effect can be considered an instance of punishment M 
ing: There is a stimulus, the taste of the food; there is a response, drinking; 
and the connection between them is weakened by the negative consequences, 
the illness, that follows some time later. Garcia himself and most oe 
gators working with the phenomenon do not accept this paradigm, anc 


: = A git p jan and 
serious question about its validity has been raised by a study of Domjar ioe 
: p A : s ; A ; for expos 
Wilson (1972). These investigators used a variety of techniques for exp 

rats to a taste cue in the 


absence of any consumption, In one Sna 
rats were curarized so that they could not drink or swallow the test H 
stance; it was simply flowed across their tongues. Then they were palmar i 
Later, when the animals recovered from the curare and illness, they pe 
a greater aversion to the flavor of the test fluid than did control animas 
that had been cur ed. Thus it appears that the Garcia 
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a food that may have been highly preferred originally as though it now 
tastes bad. This observation is consistent with the idea that the rat learns 
to expect the US when it encounters the CS. But we still need to con- 
ceptualize the learning mechanism. 

Let us consider three possible learning processes that might produce the 
Garcia effect, One view is that an S-S association is formed (Revusky, 1971). 
Although the rat is predisposed to form certain kinds of associations, such 
as between tastes and illness, during the time between ingestion and illness 
the taste cue is gradually lost, either because of forgetting or because of 
interference, According to Revusky, what makes learned food aversions 
possible after such long delays is that there is relatively little interference 
from competing taste stimuli during the delay interval so that it is possible 
to remember the cue that has to be remembered if the association is to be 
formed. This view is inconsistent with the finding that birds, which rely 
conditioned aversion learning with long inter- 


upon visual cues, also show 
ass of intervening visual stimulation 


vals, even though there must be am 
during the delay interval, Revusky appears merely to have restated the 


problem of why other kinds of stimuli do not interfere with the association 


of the relevant cue with its ultimate consequence. 

Kalat and Rozin (1973) have suggested a different mechanism. Their pro- 
posal is that the animal is initially somewhat suspicious upon encountering 
novel things. The rat shows neophobia particularly to new food substances, 
but over a period of time its suspicion is dispelled as it learns that the food 
ave beneficial consequences such as quenching thirst 
or providing calories. Kalat and Rozin tested their proposal against the 
more conventional associative-learning hypothesis by providing rats with 
different opportunities to learn about the safety of the test food. One group, 
let us call it group 4, was fed a novel substance and then four hours later 
was made ill. Group 4 was fed the same food and made ill 4-hour later. 
The critical group 4-4 was offered food twice, once four hours before poison- 
ing and again 4 hour before poisoning. According to the forgetting or loss- 

d group 4-4 should both have shown strong 


of-association theory, group 4 an 
food aversions because they each had an equally short half-hour interval 


between the last presentation of food and illness. But the results of the 
experiment were quite different. Group 4-4 showed essentially the same 
aversion as group 4 because, it was argued, both groups had approximately 
the same amount of time, 34 hours versus 4 hours, to learn that the food 
was safe. According to Kalat and Rozin, the temporal gradient (see Figure 
9.2) does not reflect a forgetting of the taste, but complementary or oppos- 
ing learning about the safety of the new food. This hypothesis explains why 
the novelty of the test food is so vitally important and why some flavors 
appear to be more salient in that they more readily lead to acquired aver- 
sions than others, Kalat and Rozin’s interpretation of the Garcia effect is 


is safe and may even h 
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somewhat reminiscent of Krechevsky's concept of hypothesis KEES 
ter 5). Upon encountering a novel food the rat appears to A 
the food may be bad, and so it cats a little and waits; it tests the pN ie 
After a time, if everything is okay, the food then becomes more TER 
and the original hypothesis is replaced by a new one that the food is go = 
There is, however, a fundamental difficulty both with the more Sees 
tional associative view of Revusky and with Kalat and Rovin’s seat ee 
tion of the Garcia effect. This problem is simply why does the asap gd 
involve specifically the taste of the recently consumed novel food? Hr see = 
to be more a matter of memory and retrieval than of input and bees olan 
Posing the question as a problem of memory, why does the animal ee 
ber what it ate at the time it becomes sick? Why, when it becomes ab i: ves 
the rat remember specifically the taste of the novel food, and why ae 
bird or the guinea pig remember specifically the sight of the food? Sait 
are these particular contents of memory retrieved and not some irrelevé 


event that occurred before or afterward? It ise 


y > 1er 
asy enough for us to see otl 
events as irrelevant, 


5 age J at can be 
but how does the rat do so? Perhaps all that ¢ 


AS 
A x i i . > 2 Pere ive so. ¢ 
said at this point is that the animal's memory is organized to do oe 
x > . oe lt: q 3 S i 
Garcia and Koelling originally proposed, perhaps these kinds of cue 
these ki 


nds of consequences 
learning occurs very easily. 

It is becoming clear that the 
us to know something 
how it solves its proble 
lems are solved by 


$ z Kre hich 
are organized into a closed system within w 


proper understanding of learning naume 
about an animal's ecological niche; we must know 
ms. It is no longer possible to assume that all prob- 
Even in 


n aarne- 
animals as capable of lear 
aterials th 


at enter into learning, the kinds 
es that become related, have an 

intricate structure of their own t 
of animal. Learning ha 
learn a particular thing 
Perhaps that is all we can take for gr 
In the remainder of this ch 
instances of failure to learn under conditio: 
ing theorist would surely have Expected le 
instances in which animals le 
been regarded as making lear 


ing as the rat, we see that the m 


4 his ae 
a particular animal cat 
it is senetically endowed to do so- 
anted at this point. 


apter we shall consider briefly a number of 


ns in which the traditional learn- 
Anne: anid @anginhey of contrary 
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tion. One difficulty in obtaining such an effect, however, is that there are 
not many states of ill health from which an animal can recover rapidly. 
The recovery from an illness is typically much slower than its onset. Thia- 
mine deficiency is an exception, in that a massive dose of the vitamin can 
overcome a severe deficiency within an hour or so. Zahorik and Maier 
(1969) fed thiamine-deficient rats saccharin and followed this procedure 
with a large vitamin injection; they did find an increased preference for 
saccharin. 

We might expect the same type of increased preference for a food that 
provides caloric value to a hungry rat. Such a mechanism, a “positive Garcia 
effect,” would lead to animals having a convenient preference for foods of 
r such an effect has not been reported. A pair 
of studies recently conducted at the University of Washington suggests one 
reason why a positive Garcia effect may be difficult to find. In one experi- 
Parker, and Skorupski (1974) gave two groups of rats 
a coma. For one of these groups 
ty water; these animals 


high caloric density. But so fa 


ment, Weisinger, 
insulin injections sufficient to put them into 
associated with insulin shock was 
subsequently showed an aversion to the saline solution. But the second 
group of animals, tested with saccharin, failed to show an aversion to it. In 
two other groups gastric distress was produced by formalin injections. Again 
the animals w tested either with a saccharin solution, which did produce 
a conditioned aversion, or a salty solution, which failed to produce a learned 
aversion. Weisinger and his colleagues appear to have discovered a con- 
straint upon the possible relationships between the cue and the consequence 


that can be learned. Their explanation is that insulin not only produces a 


distressing: comatose state; it also produces a need for sugar and a specific 
appetite for sugar. Formalin, on the other hand, produces a salt imbalance 
such that the animal needs and will seek out salt solutions. What the experi- 
ment appears to show, then, is that if the toxic agent produces a specific 
kind of need state and if the animal has an “innate recognition mechanism” 


for dealing with that need, 


override any aversion that might otherw 
into insulin shock, the animal develops an increased preference for sweets 


that overrides any possible learning to avoid the sweet flavor. In effect, the 
animal's innate recognition mechanism replaces the learning mechanism in 
the solution of this particular problem. It is as if the animal made ill by 
insulin cannot attribute its illness to the prior consumption of a sweet sub- 
stance and as if an animal made ill by formalin cannot attribute its illness 
salt solution. 


the novel food 


then this innate recognition mechanism will 
ise be learned. After having gone 


to the prior consumption of a 
To return to the question of why it is so difficult to obtain increased 


preference for foods associated with caloric benefit, perhaps the rat has an 
innate recognition of high-calorie foods like fats and sweets and because of 
this innate recognition mechanism there is no need for a learning mecha- 
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ypothesis indirectly e 
produced in rats an “artificial” need State for ee 
evolved an innate recognition mechanism, the a ails 
produced by morphine withdrawal. Rats were first regularly mjeried te 
morphine until they became physiologically dependent. Morphine i rA San 
withdrawn until the animals became ill. Then, just before being a ea 
Maintenance dose of morphine, the animals were fed a saccharin Bo # rel 
Compared with controls, these animals quickly developed a persevere at 
saccharin. One implication of this research is that the rat Eaa i ema 
systems for Solving its general nutritional problems. First, it od z for 
recognition mechanisms such that a certain need produces an appes në 
the needed substance, Then, to take care of the multitude of Gases peng 
Conditions of need, Particularly illness, lead to no innate recognition, 


. s P j as an we have a further 
is the learning mechanism. If this analysis is correct, then we have a 
structuring of what an animal can le 


lar foods and the cons 
the relationship betw 
makes such learning 
effect seems to show i 
also be met before s 

Hogan (1973) pos 
food. A young chi 
the ground: food, 
grows older 


‘arn about the relation between bagi 
equences of ingesting them. Garcia showed tliat iy 
cen particular cues and particular conseguendo. a 
possible, and what continued research on the Ga oat 
s that some further, rather specific requirements mus 

uch learning occurs, 
ed the question of how 


ie ize 
chickens learn to recogn 
ck approximatel 


y three days old pecks at anything = 
s, shadows, anything. But as the chic! 
s pecking and pecks per 
8ravel are still taken in and are 


to which Hoga 
piles of sand and seeds whos < Properties were such that the chicks 
pecked approximately equally often a 


and food, that is, there was NO 
d one group of ch 
Approximately an hour Ja 


seem to learn nothing ab, 


nd. Given 
they are food. The mos 


sand, the birds thus 
en seeds, th 


ey seem to learn that 


UP was given only s 1 then 
i i y Sand and 

fed a high-calorie meal through a tube directly into hak When 
tested an hour later, these chicks showed a Strong x 

It appears that the chicks show 


Preference for the sand. 
i on the subsequent test for 
ave deriveq Some nutritional 


a preference 


whatever it was they ate, Provided that they 


benefit during the interim, 
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One of the most interesting findings reported by Hogan is that a delay 
of about an hour between the initial exposure and the subsequent test is 
necessary if the preference is to develop. Here, then, we have an unusual 
form of learning, which occurs only when the consequence is considerably 
delayed after the cue is presented. From a nutritional point of view, of 
course, the necessity of such a delay is reasonable. But from the point of 
view of traditional learning theory, we find an anomalous inverted delay 


gradient such that, the longer the delay, up toa point, the better the learn- 
ing. Hogan has found that there are innate recognition mechanisms in 
addition to the learning mechanism just described. For example, if a chick 
is offered a highly palatable mealworm, it shows a strong and immediate 
mealworms over either seeds or sand. It appears that the 


preference for 3 
efits of seeds but that it recog- 


chick must learn about the nutritional ben 
nizes, probably from the taste, the value of mealworms. 
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An interesting case where we might have expected learning yet find none 
was reported some years ago by Schneirla (1933). Schneirla had found ants 
capable of learning in various situations, so he tested them on a maze 
problem. The starting point was placed at the ants’ nest, and food was 
placed in the goal box. When the ants left the nest to forage for food, they 
would wander through the maze, but Schneirla found that they showed 
little or no learning of the maze pattern; there was no reliable decrease in 
errors. Does this negative finding mean that ants cannot learn mazes? Not 
at all. Schneirla found that if the maze was turned around so that the ants 
had to run through it to carry food back to the nest, they quickly learned it. 
The ants’ ability to solve the maze pattern depended upon their motivation; 
they could learn how to get to the nest but not how to get to the food. 

It could be said, of course, that ants are simple-minded animals; cer- 
tainly there would be more generalized learning ability among vertebrates. 
Better perhaps, but not infallible. Kramer (1957) was working with starlings 
that he kept captive in a large enclosure. They roosted in the center of the 
enclosure, and he trained them to go off each morning to a particular part 
of the enclosure, for example, the eastern side of it, to obtain food. The 
Starlings quickly learned the problem. But Kramer, who was primarily con- 
cerned with navigation in animals, evidently became concerned with whether 
or not his birds were really navigating. The sun was in the east in the 
morning, so perhaps they were merely flying toward it (and the birds trained 
to go to other parts of the enclosure might be flying at some fixed angle 
to the sun). The critical test, of course, was to move the sun. Kramer did 
this by testing the birds in the afternoon. The results showed that the birds 
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were not ap 
flying to th 
his birds t 


proaching or orienting with regard to the sun but were Pa 
€ eastern part of the enclosure. Kramer then attempted io. ies 
© approach or orient with regard to the sun by feeding then bie 
n the morning and the west in the afternoon. He found ini sie 
Starlings had great difficulty learning this simple problem. Here we omen 
irony of an experimenter using the most splendid stimulus in the s Win 
system and finding his animals apparently unable to learn simply to ah 
proach it. In a series of further experiments Kramer demonstrated that es 
Starlings not only noticed the sun’s location but were actually a i 
information in conjunction with their own biological clocks pe Sept 
the appropriate direction. It was as if the birds made a calculation invo 


i ; $ sed this 
the location of the sun and the known time of day and then us 


5 . t 
à p 7 ; ; ` : ; rection. Bu 
information to navigate quite accurately in a given compass direct 


; 2 x rever it happened 
the much simpler task of simply approaching the sun wherever it hap] 
to be was not so easy to learn. 

It could be said tha 


: A r : s: surely better cue 
t birds are simple-minded animals; surely bette 
utilization would 


be shown by mammals, However, Konorski (1907); i 
> of studies with dogs that show serious constraints ri 
best friend. One experiment required the dog to 8° a 
ward and approach food upon receiving a given signal and not to app! “he 
food in the presence of an alternative signal. The two signals were audito 2 
stimuli that differed in quality; one was a tone, and the other was a buzzer. 
The dogs had no difficulty learning this so-called g 
But then Konorski used the same two cues to 
would contain food. One signal indicated f 


rare 4 > could 
the other indicated it would be on the left. He reported that the dogs cou 
not solve this problem, It might be argued that the T-maze problem was 


inherently more difficult than the §0-no go problem. But this was shown 
not to be the case in two xperiments, 


T-maze task were two tones, one placed di 


directly below the dog, each tone signifying on which side of the T maze 


the food would be located in a given trial, Now the T-maze problem was 
solved with little trouble. But when the sx, 


A : same Spatially differentiated cues 
were used in the go-no go discrimination, this task became unsolvable. We 
are forced to the conclusion th 


i at spatially localizeq cues are effective in the 
solution of Spatial problems but not for 


à ni ‘ the 8°-no go discrimination. 
whereas auditory signals of different qualities are effective for a go-no go 
discrimination but not for simple spatial learning, § 

The skeptic could Say that even the do: ot SO smart, and when it 
comes to the proper utilization information T 
question of the human’s ability. But this : 2 
Recall that Watson had conditioned fez 


iscrimination. 
O-no go discriminatior 
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ri ar 
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noise as the US and a white rat as the CS. A careful replication of this 
study was reported by Bregman (1934). She was able to obtain the same 
kind of fear conditioning with an animal as the CS, but when she used an 
inanimate object such as a block or a bottle as the CS she found no fear 
conditioning. The infants were reported to be unhappy about the condi- 
tioning experiment, but the tremendous conditioning of fear reported by 
Watson and Rayner and widely accepted as a model for all fear conditioning 
was found to be restricted in the young human subject to animate things. 
Even a wooden animal will not do (Jones, 1924). 

We may suspect that little Albert was very much like Kalat’s rat in that, 
on his first encounter with a strange animal, he was suspicious of it. Perhaps 
humans are neophobic about certain classes of events, but if everything 
turns out all right they learn that they are safe. But let there be some dis- 
turbance, and fear will be quickly learned. We may suspect, too, that in 
the absence of this initial suspicion it may be quite difficult to condition 
fear, Seligman (1972) has made a similar analysis of phobias in adults. It 
turns out that virtually all the millions of people suffering from phobias 
share just a few specific fears, restricted to certain narrow classes of events. 
Phobias nearly always pertain to animals, snakes being the most common, 
and certain situations, being closed in or falling off high places being the 
most common. None of these millions of people has a phobia about coke 
bottles, door knobs, ash trays, or any of the other multitude of inanimate 
things that crowd our everyday world. Seligman therefore calls into ques- 
tion the notion that phobias develop as a result of classical fear condition- 
ing, as had been said by Mowrer (1939) and accepted by most other writers. 
Another reason for skepticism is that the people who have phobias about 
snakes or high places have never experienced an appropriate US; that is, 
they have not been bitten by snakes or fallen off high places. The fear of 
such events is incipient, or latent. It is probably triggered as much as any- 
thing by cultural factors. There is thus every reason to believe that even in 
the all-learning human, who can learn to go to or from, who can use the 
sun either to approach or to navigate with, who can use all kinds of audio 
signals for all kinds of purposes, there are biological considerations that 
govern how likely he is to learn about certain classes of stimuli. 

All the examples th 


or special facilitations of learning about the relationships between cues and 
consequences. To the extent that these specific relationships give structure 
to what is learned, they impose a constraint upon the generality of the 
Pavlovian paradigm. But there is another large class of constraints in learn- 
ing that has to do with the ability of a particular animal to be controlled by 
particular reinforcement contingencies. In the remainder of this chapter, 
we shall look at some of these constraints and special facilitations. 


at we have discussed so far illustrate constraints upon 
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Constraints on Response Acquisition 


Keller and Marion Brel 
of animal behavior for con 
ing unusual behaviors in 
advertising agency or 


and made a business of training unusual forms 
nmercial purposes. They made their living shap 
unusual animals that might be desired by = 
for the movies. The Brelands used at aE 
operant techniques, often involving considerable shaping to S 
desired response and often involving the chaining of simple behaviors RIA 
complex sequences. The standard operant techniques were generally A 
cessful but not always. In a paper entitled “The Misbehavior of A EEA 
(Breland & Breland, 1961) they describe troubles they had fulfilling a ie 
tract that called for a pig to put pseudomonies (wooden disks) into a a 
bank. The pig was intelligent, friendly, and it loved to eat. The trou : 
began when the Brelands required the pig to drop a coin in the slot ne 
obtain food, Everything went awry; the pig began tossing the coin in n 
air, snuffling and snorting, rooting on the ground, tossing and rooting, an 
so on instead of dropping the coin in the bank. The misbehavior was RK 
serious that further training was abandoned. The Brelands decided Ea use 
a raccoon. Raccoons are extremely dextrous, highly intelligent, and just = 
cute putting money in the bank as pigs are. There was no problem on. 
continuous-reinforcement schedule, but as Soon as a two-to-one ratio was 
required the raccoon’s behavior deteriorated, It would take the two oer 
and instead of depositing them in the slot would dip them in and out anh 
rub them together, as raccoons do when they are wa hing their food. This 
behavior was so persist ad to be given up. 

‘The Brelands describe another disaster that occurred when they attempted 
to train a chicken to play baseball, The game w 
pinball machine. The idea was that the ball woul 
and the chicken would pull a chain to swing 


was hit out through the outfield, a bell would ring, and the chicken could 
run off toward first base to get its food. Jt was a fairly simple matter of 
chaining, and everything went well; the chicken was pullin the chain and 
running to first base, and everything wa r until Be et was intro- 
duced into the game; then—chaos, the chicken Tit the ball, 
instead of going off to first base t l it would attempt to field 
the ball itself, pecking furiously at it, and chases, He ball all 
around the diamond. The baseball g be Manos 1 

What is the explanation of these dramatic failures af l mS and all 
this misbehavior? Notice that Some segments of the desire 7 sel a A 
readily obtained but that at some point the reinforced bekia ote ee 


and was replaced by each a l 
p od-getting behavior. Each 


ent that the project h 


as arranged on a sort of 
l roll toward home par 
a little metal bat; if the bal 


© collect its fooc 


flapping 
ame had to 
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animal had a token that represented food and could be exchanged for food, 
but because the token did represent food each of the animals began to act 
toward it as if it were food. The pig rooted around with its food token, the 
and rubbed it, and the chicken pecked it. In each case the 


raccoon dipped 
ng behavior intervened and interfered with 


animal's species-specific food-gettir 
the behavior that the Brelands were attempting to instill. In some of the 
failures that the Brelands described, the interfering behavior become so 


strong that the animal was completely cut off from further reinforcement. 


So, whereas animals can frequently learn unusual food-getting behaviors, 
the innate relationships between particular behaviors and particular conse- 


quences can supersede, preventing the reinforcement contingency from 


controlling the animal’s behavior. 
Another illustration of the same type of constraint appeared in Stevenson- 


Hinde’s study (1973) of learning in chaffinches. She found that she could 
train these birds to perch on a specific part of the apparatus, which was 
arranged to produce the recorded sounds of chaffinch songs. She also found 
that chaffinches would learn to peck a key to obtain food. But at the same 
time she had great difficulty training them to perch in a particular location 
to obtain food and had no success when she attempted to reinforce pecking 
a key with song. Evidently pecking for food is an appropriate food-getting 
response, and perching ina particular place is an appropriate response for 
producing song, but it is very difficult to interchange the parts of these two 
systems. re G 

The Dutch ethologist Sevenster (1973) had similar problems in producing 
learning in the stickleback fish. These animals are highly aggressive fighters 
under appropriate conditions, and males will chase off other males in a 
characteristic manner. They are also highly aggressive lovers and will give 
characteristic courting displays to female sticklebacks. Sevenster then made 
the presentation of either the male to fight or the female to court con- 
tingent upon one of two different operant responses. One response was to 
bite at a glass rod inserted in the tank. He found that this response was 
rapidly learned when reinforced by presentation of the male, but it was not 
learned when the female was presented. The second operant required the 
fish to swim through a wire hoop. It was found that the stickleback would 
quickly learn this response when it was reinforced by the presentation of 
ot when the attempt was made to reinforce it by presenting 
to be two separate systems, an aggression 
Sevenster analyzed the consummatory be- 


the female but n 
the male. Again, there seem 

system and a courting system. 
haviors for each system and concluded that the constraints on learning could 
be characterized in the following way- If the operant response is a part of, 
closely resembles, or is compatible with some part of the consummatory 
reaction, then it is learned. Otherwise it is not. Biting the rod is compatible 
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with aggression, so it is learned as 
responses must be inhibited 
response involves swimming 
manner, so it is e; 


zs pe nes ressive 
an operant, but in courting aggressi i 
iti 3 A rtin: 
» and so biting is not learned. The FON at 
e a Cteris Z187 
toward the female in a characteristic zig 


x » ma wever, 
asy to go through the hoop; attacking the male, es ee 
i i $ reep se jis ear N 
requires a different kind of approach, so the hoop response is not 

In all these failures of learning w 


e can see that the animal was pan 
to perform an Operant that was incompatible with the ancora 
response. What can we learn about learning from situations in whic s on 
operant is highly compatible with the consummatory response? One of he 
most interesting and most widely investigated situations of this type $ e 
so-called autoshaping experiment. In Chapter 7 we mentioned ese? fae 
customary procedure for shaping bar pressing in rats. Essentially the eal 
techniques are used to establish key pecking in pigeons. Briefly, the ao 
is first magazine-trained; then from time to time the plastic key 38 TRA 
nated, and the experimenter gradually shapes orienting, approaching, ie 
tative pecks, and finally outright pecks at the illuminated key, using S 
sound of the feeding magazine and food to obtain the desired iy ee 
response. The entire Process can be carried out in an hour or arbi i 
experimenter is careful and does not reinforce inappropriate behavior, i 
1968 Brown and Jenkins reported a technique that tremendously facia i 
the entire process. What they did was simply give animals free food, that a 
food presented without regard to the birds’ behavior, every fifteen seconds, 


. » š same 
and they illuminated the key just before food was presented. In the sat 
amount of time that tedious hand shaping ordina 


were busily pecking the key and obtaining 
had shaped themselves or had become ‘ 2 
There are a number of ations of autoshaping. Within 
the Skinnerian framework, it may be supposed that pecking the illuminated 
key has a very high operant rate, so that there is a high chance that the 
reinforcement contingency wi with the behavior. Within the 
Hullian framew " Because the illumination 
» it must become an incentive 
$ I's MOtivational] level and further in- 
creasing en tries level of key pecking. But such interpretations would 
ear to be excluded in a iea an 
liarn (1969). They fies autoshaping Study reported by Williams and 


by Brown and 


rily requires, their ae 
food. During the session the bird 
‘autoshaped.” 

possible interpret 


í ation 
of the key, but whenever a key peck occur y d by Heata i je 
food. Thus, whatever incentiv, vati ted the delivery 


producing the behavior ' factors were initially 
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ported obtaining substantial amounts of key pecking; in some birds the 
rtually all food reinforcement was elimi- 


response became so strong that vi 
nated, The Williams’ phenomenon, which they labeled “negative auto- 
appears similar to the behavior of the Brelands’ animals: A 


tting behavior becomes so strong that it totally over- 


maintenance,’ 
species-specific food-ge 
rides the reinforcement contingency. 

We have become accustomed to thinking of the key-peck response as an 
operant and as an arbitrary, convenient, easily measured response whose 
strength can be controlled by its consequences. But we can see, in the auto- 
shaping procedure generally and in the negative automaintenance proce- 
dure particularly, that the strength of the response is not controlled by its 
consequences. It is evidently controlled by the signaling properties of the 
illuminated key. 

Further doubt about the validity of the conventional operant interpre- 
tation of key pecking has been raised by Moore (1973). Before describing 
Moore's results, we should note that the consummatory response in the 
pigeon is unique in one respect: The pigeon does not drink as most birds 
do, by filling its bill, tossing its head back, and letting the water run down 
its throat; it drinks by giving a series of bill movements that is evidently 
equivalent to licking in mammals. The bill makes a series of rather gentle, 
comparatively slow, and relatively prolonged movements. The eating re- 
sponse, on the other hand, consists of a single sharp, very fast, and rather 
forceful stroke of the beak. There are other parts of the total pattern: for 
example, the eating response is executed with the eyes shut and the beak 
opened to seize food just at the moment of contact. Moore autoshaped 
pigeons under conditions in which one key color signaled food and another 
key color signaled water. His birds were both hungry and thirsty and were 
given free food and water intermittently in association with the appro- 
priately colored key. High-speed photographs of the birds’ autoshaped 
responses show that the pigeons were, in effect, eating the food key and 
drinking the water key.4 Moore’s (1973) interpretation of these results is that 
the key-peck response is not really an operant at all but is a classically 
conditioned consummatory response. In his experiments, a particular key 
color becomes a discriminative CS. which is paired with a particular US, 
and the appropriate consummatory response simply becomes classically 
conditioned to it. 

Moore's interpretation can be ay 
nomena that have been previously d 


pplied to a number of the other phe- 
escribed. For example, the misbehavior 


4 The same finding about the different topographies of pecking for food and pecking 
for water under- normal conditions of reinforcement had been briefly noted earlier by 
Wolin (1968), but the phenomenon did not attract any attention until Moore rediscovered 


it in the autoshaping situation. 
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of the Brelands’ animals can also be attributed to the fact that 
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ai 5 l tokens b 
tory or food-getting behaviors become conditioned to the fooc 

cause the latter are CSs for 
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food. Moore’s peculiarly conservative 
conditioning as involving 


CS éan 
attachment of a replica of the UR to E 
be modernized along the lines indicated in Chapter 8, Arenan 9 of his 
the force of Moore's argument or detracting from the pA aa ain modern 
data. We may think of the illuminated key as a Cs in the CHF whet 
sense that it predicts food (or water) or provides A Tane a 
it will be forthcoming. The key has the additional proper a ya pecked. 
highly localized discrete object that lends itself admirably to SA ead pre- 
The same point can be made about the different food tokeny, > cule 2088 
dictors that so disrupted the behavior of the Brelands’ animals: x ; sec 
the animal was able to direct its species-specific food-getting behavior 


‘ Jey (1974) indi- 
the token. A recent study by LoLordo, MacMillan, and Riley (1974) 
cates that the more localized the food predictor is 
food, the more likely the 


autoshaping phenomen 


and the better it E 
pigeon is to peck it. In short, it appen 3 pre 
On requires, first, that the key or whatever 5 a be 
dictor of food and, second, that it have stimulus properties such tha 
animal can direct food-getting behavior at it, 

The whole series of experiments th 
are a number of 5 
ineffective in contr 


at we have described indicates that ne 
h operant-conditioning procedures si 
olling operant behavior, Many of these situations ae $ 
to share a common chara isti require the animal to make k 
response that is different į ant consummatory respons : 
Then the autosh ates that there are other situation® 
in which, again, oper 8 procedures are ineffective in a 

and these Situations appear to have the chara 
teristic that the required opera i 


ituations in whic 


summa- 
ike the relevant consum of 
» just what is the domain ly 
` zi n 

? Is the Operant paradigm © 


r . ` i ie 
plica and to motivational systems in which ther 
is some flexibility of the consummatory and food-getting behaviors? 


Peter Covey, working at the Seattle 700, has made some interesting 
observations about the flexibilit ummatory behavior, A problem 


at the zoo was that the ni rdeiles minor) would not eat food 
from a dish. Normally awk picks UP insects by flying aron 

eed the animal in th v 
ood over its } 
rocedure w; 
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but the bird did not eat enough this way, so the 
original problem had not been solved. The next step was to present morsels 
of food, not in the shape of a ball, as they had been up to this point, but 
about the size of a cricket. Eventually Covey was able 
daily ration from cricket-sized pieces of 
behavior that is never seen in the 


a dish on the ground, 


in a rolled-out shape 
to get the bird to eat its entirely 
food in a dish, a kind of consummatory 
wild nighthawk. 

Here, then, we appear to have an inst 
ction with contingent reinforcement, 
But there are other possible interpretations, There is the 
amatically split the once-unified Hullian theo- 


ance in which a shaping procedure, 


used in conjun was effective in chang- 


ing a behavior. 
basic question that has so dr: 
rists: How much of the change in behavior can be attributed to the 
response-reinforcement contingency per se, and how much of it should be 
attributed to learning about the altered S-S relations that occur during train- 
ing? During the course of training was the response really being modified 
or was the perception of food being modified? Did the 
responses, Or did it consist primarily of the estab- 
r symbols? However the question is answered 
in this particular instance, it is clear that much of what passes for operant 
conditioning consists of changes in the perception of the stimuli that control 
the animal's behavior. In the Skinner box the original effect of illuminating 
the key is only to produce some orienting behavior or investigatory reflex. 
But eventually the key appears to become virtually synonymous with food, 
so that, as we have seen in Moore’s research, the pigeon will literally eat it. 


by reinforcement, 
learning consist of new 
lishment of new food tokens © 
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Avoidance learning has always been a great challenge to the reinforce- 
ment theorist. One of the peculiarities of the avoidance procedure is that 
when the avoidance response occurs there may be no explicit event that can 
be pointed to as the reinforcer for the response. By definition, the avoidance 
hock from occurring, at least with most avoidance proce- 
dures, so we cannot point to the shock as a potential source of reinforcement. 
On the other hand, it is not satisfactory to cite the nonoccurrence of shock 
as a reinforcer because logically there is a host of things that do not occur, 
and the behaviorist is hard put to say why not being shocked should be 
relevant in the avoidance situation, whereas, say, not being stepped on is 
irrelevant. The problem is eased somewhat if a warning stimulus, or CS, is 
introduced before a scheduled shock, particularly if it is arranged so that the 
response terminates it. Recall that Mowrer and Lamoreaux 


response prevents S; 


avoidance A 
(1942) found that CS termination facilitated learning. Under these condi- 


tions the termination of the CS can be and usually is cited as a source of 
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` also 
reinforcement (the reduction in fear that is conditioned to the CS can a 
be, and usually is, cited by those of the Hullian persuasion). E NONE 
But this explanation does not entirely solve the problem beeaus Pe NE 
and Lamoreaux also found some learning when either a E E mle 
procedure or a prolonged-CS procedure was used. The theorist is ae CS 
obliged to hypothesize the existence of some internal and E REG 
whose termination is reinforcing. The explanation of avoidance a 
then depends on a number of hypothetical internal events, APA ae 
classically conditioned fear, which is Supposed to motivate the resp 


ve as 
i RE eve to serve 
provide stimulus control for it, and some other stimulus event 

a source of reinforcement. 


Another approach to the problem of w 


itions 
is to raise the pragmatic question: What are the experimental ssi att 
that control it? What are the contingencies that produce ayoihmte: bler 
ing? Kamin (1956) appears to have been the first to approach the Pe 
in this way, luded four groups of rats, all of which aes 
to another in a shuttle box, For one gro 


k sing them 
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- In the second grou 


oidance response) 
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tingency performed vi 
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cre the two that were Sears 
* One group could avoid shock Td 
€r could terminate the CS but cou 
n’s results showed that animals with both cone 
and performed well. The group with neither ee 
ery badly and evidenced little learning. The group» 


Pi n 5 iva- 
ency showed intermediate and approximately equ! 5 
lent performance. It could be concluded therefore that there are two effec 


tive contingencies: one the avoidance of shock and the other termination of 
the CS. The conventional view was exonerated; Cg termination did make 4 
contribution. The other troublesome factor, the avoidance of shock, could 
be handled by Supposing that reinforcement is not dependent upon avoid- 


ance itself but that competing behavior that fails to avoid is punished by 
unavoided shocks. s 


Bolles, Stokes, and Younger 
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behavior). We found that all th 
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design using a different avoidance-learning situation. In the second experi- 
ment rats were required to run in a wheel. Again the different contingen- 
cies were either allowed or not allowed for different groups, so that their 
individual contributions to running-wheel avoidance could be determined. 
The results indicated that escape from shock made very little contribution, 
and termination of the CS made very little contribution; the all-important 
factor in this experiment was the avoidance contingency. At this point it 
began to appear that an analysis of avoidance behavior, in terms of the 
experimental contingencies that control it, might not have any general 
answer because the relative effectiveness of the different contingencies de- 
pends upon what the avoidance response is. 

Subsequent research at the University of Washington has confirmed this 
conclusion. In the one-way situation (which is like the shuttle box except 
that the animal is returned to the same side of the apparatus between trials, 
so that the running response is always executed in the same direction rather 
than back and forth) the acquisition of avoidance appears to be wholly 
uninfluenced either by CS termination (Bolles & Grossen, 1969) or by the 
escape contingency (Bolles, Grossen & Hargrave, 1969). More recent and 
still unpublished results from our laboratory indicate that learning of bar- 
press avoidance is actually impaired when rats are given the escape contin- 
gency and that neither the CS-termination contingency nor the avoidance of 
shock appears to have any consistent effect. Although rats are generally poor 
at learning bar-press avoidance, some do learn and those that learn do not 
seem to be benefited by any of the experimental contingencies that we so 
o produce learning. In short, our investigation of avoid- 
ance learning in a variety of different situations requiring different responses 
from the animal indicates that there is no consistent pattern of experimental 
contingencies responsible for the learning. A further problem in such an 
analysis of contingencies has been noted by Bolles, Moot, and Grossen 
(1971): The contingencies that appear to be important in maintaining avoid- 
ance behavior and preventing its extinction are not necessarily the same as 
those that are responsible for producing the initial learning. Whole new 
patterns of contingencies turn out to be important in extinction, at least in 
the shuttle box, which is the only situation that has been extensively studied. 
The pattern of controlling contingencies just does not seem to be coherent 
enough to have much explanatory power. 

The most obvious and impressive thing about avoidance learning in 
different situations is that in some situations the avoidance response is ac- 
quired in just a few trials, whereas in other situations learning appears 
barely possible and in many instances really is impossible. To put it frankly, 
the average rat does not learn to press a bar to avoid shock. A few animals 
learn, but they typically perform inconsistently, and giving them extended 
training does not always help. Over the years a variety of tricks have been 


carefully arrange t 
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i i inimizi amount 
discovered that provide some relief (for example, minimizing the 


ricks can be 
of shock improves performance somewhat) but none of these tricks 


i i emam a very net 
counted on to give good bar-press avoidance, It remains a very 


i: ri - well in 
problem for the rat. By contrast, most rats learn to perform rather 


the shuttle box, A group may avoid on 80 percent of the trials after ell 
200 training trials, Running in a wheel js learned even faster, ee 
taking only forty trials, and performance typically reaches an (vis ak 
close to 100 percent, Then there are situations such as one-way avoidé 


. 4 3 4 R 3 ; ~ box—in 
task and Situations in which the animal jumps out of the shock 
these situations some learning 


Š rial, and the 
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yi ras i 
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vidence some learning in some situations a 
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no indication of learning in another situation 
after 1000 trials, k that 
It is a serious indictment of our conventional theories of learning er 
they provide no hint that the choice of response is so important. The mos 
casual observation of rats in difference avoid 
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ance-learning situations reve 
required of the anim 
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parameter. It is vast] 


the manipulation of 


al is a tremendously aan 
y more important than the analysis of essing lasek 
other experimental conditions, the prior expenence be 
the animals, or various kind of physiological interventions that might E 
made. Clearly, then, the determination of what the avoidance apo 
shall be is not just a matter of convenience; it js the all-important considera 

An attempt to explain these vast differences in ease of learning was pre 
posed in a paper entitled “Species-Specific Defense Reactions and Avoid- 
ance Learning” (Bolles, 1970). The argument is that, just as animals have 
species-specific behaviors for obtaining food, mating, and solving other bio- 
logical problems, so they must have innate defensive beh Such be- 
haviors must be innately organized because na 


tunity for the animal to learn to avoid pred 
A small and relatively defenseless a at cannot afford to learn 
to avoid these hazards: it must hay a sive Béeliaviare that keep it 
out of trouble. The course, to other animals. 
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SSDR hypothesis states merely that animals have characteristic defensive 
behaviors and that avoidance learning in the laboratory is possible only 
if the response that we require of our animal is an SSDR or is at least 
highly compatible with the animal's SSDR repertoire. What does the rat do 
when it fails to learn the response that we require of it? It misbehaves. The 
frightened rat, much like the Brelands’ hungry animals, displays its own 
species-specific behavior; it freezes. 

We will continue the analysis of avoidance behavior, emphasizing the 
SSDR concept, in Chapter 10. Here we must sum up to see if we can draw 
some general conclusions about the structure of learning. It should be 
apparent that the operant behaviors that have been most widely studied 
in the laboratory, key pecking for food by the pigeon and running in a 
shuttle box or pressing a bar or traversing a maze by the rat, are not just 
typical and convenient behaviors. They are behaviors that these particular 
adept at learning, and they learn them at least as 


animals are especially 
haviors are part of their natural means of solving 


much because these be 
particular problems as because they are reinforced. , 

We have surveyed a variety of learning situations in which the behavior 
does not seem to follow the generally accepted rules of the basic paradigms. 
We have noted two kinds of constraints: constraints on the Pavlovian 
paradigm, in which only certain s-S* relations can be readily learned, and 
constraints on the reinforcement paradigm, in which only certain R-S* rela- 
tionships can be readily learned. Often, of course, these paradigms do work, 
and we must be thankful for that. But we must also wonder why they work 
when they do. Some writers (for example, Seligman, 1970), have suggested 
that there are no general rules of learning; Seligman proposes that because 
of its genetic endowment, a particular species of animal may be “prepared” 
to learn one association and quite “unprepared” to learn another. But what 
we have seen of the structure of learning suggests an even more profound 
problem with the basic paradigms. It suggests that the idea of a learned 
association may not be useful. We have to take a new look at learning, 
keeping in mind our new discovery that learning seems to depend upon an 
interaction between an animal's genetic constitution and some fundamental 


property of the learning procedure that, apparently, we have not yet fully 


grasped. 
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The early experiments on the Garcia effect have been reviewed by Revusky and 
Garcia (1970), but & lot has happened since then. Rozin and Kalat (1971) present a 
valuable analysis of the Garcia effect and related food-preference phenomena from 
a somewhat different point of view. Various constraints on learning are described 
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in two recent collections of papers: Seligman and Hager (1972) present a number 
of the original studies, including some of Garcia's work, while Hinde and Stevenson- 
Hinde (1978) present a number of more recent studies and more extended analyses 
Much of the research mentioned in this chapter can be found in these two see 
In addition, Shettleworth (1972) has made a detailed analysis of the structure © 
learning in different species of animals. An introduction to the avoidance-learning 
literature can be found in Bolles (1975). 
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In this last chapter an attempt will be made to synthesize the various 
parts and pieces of the total learning puzzle. It may have seemed in earlier 
chapters that we were close to putting some major parts of the puzzle in the 
right places, but in each case we found serious problems. There were prob- 
lems with the early formulations of Thorndike and Watson and with the 
more contemporary formulations of Hull and Skinner. There are problems 
with the current tendency to view procedures for producing learning as 
ortant than the theoretical underlying processes. Although in 
some respects this approach is a clear advance over some earlier theoretical 
statements, it is not entirely satisfactory because sometimes the procedures 
work and sometimes they do not. The learning theorist must almost neces- 


more imp 
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sarily be an optimist. He must be guided | 
manage a clearer view of what is 
he will know why 


by the hope that, if only he can 
going on, everything will make sense, and 
ng paradigms are only sometimes effective. 
approach has engendered considerable op- 
e interpretation of behavior that makes use of tenps 
such as expectancy and emphasizes the perceptual determinants of Pa 
Although many of the basic concepts are derived historically from the wor 

of Edward Tolman, a great deal of this new thinking is derived from con- 


temporary research, The theoretical synthesis that we are now about tO 
undertake is not really associated with any one theorist or any one body oi 
research. The present author subscribes to it and has made some contribu- 
tions to its formulation, but I cannot lay claim to it as my own because @ 


number of other Psychologists also subscribe to it and have contributed to it- 
Perhaps it reflects the spirit of 


the times, just as Watson's frankly mechanis- 
tic approach reflected the thinking of his era and Hull's deductive approach 
ckground of thought during the 1940s. At any rate, 
psychology appears ready for a new kind of cognitive learning theory. In 
this last chapter I will present My own view of how this new cognitive 
psychology is taking shape. I will take the further liberty of beginning the 
presentation by describing some more of my own research on avoidance 


the basic learni 
In the last few years a new 
timism. It is a cognitiv 


reflected the common ba 


learning. 


Avoidance Behavior 
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learn the running reponse. Figure 10.1 x lat they should easily 


hows 

J : S that thi . ; 

the response rapidly and in a few trials was en ota did es nia 
aching 100 percen 


problems for co 


ntemporary views of learn- 
at some avoid ‘ 


“ay 
ance responses are very rapidly 
ly, if at all, We will consider 
ailure of learning and one that 


` a 
to discover a commo! 


ina ru 


performance. 


Avoidance Behavior 195 


100f run 


3 
8 
c 
S 
3 
5 
è stand 
S 50+ 
ž 
S 
g 
5 
È 
— 1 
at ae A 5 6 7 8 


8-Trial blocks 
Fig. 10.1 Rats learn to run to avoid shock, but they 
do not learn to stand on their hind legs. (From 
Bolles, Avoidance and escape learning, Journal of 
Comparative and Physiological Psychology, 1969, 68, 
355-358. Copyright 1969 by the American Psycho- 
logical Association. Reprinted by permission.) 


ngencies were in effect, but the response 


requirement was changed. These animals were required to stand on, their 
hind legs to avoid shock, terminate the CS, and escape shock. This group 
showed no learning (see Figure 10.1). Indeed, when training was extended 
beyond that shown in the figure, the response gradually disappeared. It 


might be argued that running in the wheel is a more natural response than 


standing up, that standing up has a low operant rate in this apparatus, that 
standing up is not an “available” response 1n the frightened rat. But we 


notice that on the first few trials both responses had essentially the same 
high incidence, both occurring on about half the initial trials. The failure 
to learn by the “stand” group thus cannot be dismissed as a failure of the 


it must be a failure of r 
its consequences or a failure of learning. 


For the second group all the contir 


response to occur; einforcement, a genuine failure 


of the behavior to be controlled by i 
Now in its original form the SSDR hypothesis (Bolles, 1970) stated that 


there are innate defensive behaviors that may either compete with or 
facilitate a learned avoidance reaction. This idea gave us some purchase on 
differential rates of avoidance learning but no way to handle a failure of 
learning. We could explain why running away (an SSDR) is quickly learned, 
as in one-way avoidance, and why freezing (another SSDR for the rat) 
appens in bar-press avoidance, but we could not account for 


governs what h 
ding up, that occurs but is not learned. 


a response, like stan 
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are contracted, that matters 


box, and the one-way situation would not be so different, The oo 
consideration seems to be the functional property of the behavior, mee ; 
accomplishes, and specifically whether it permits the animal to get N 
Consider again the animals that were required to stand up in the abate 
Let us imagine that there was a small window about eight inches a ip 2 
the floor of the wheel and that if the animal stood up it could a ee 
window, pull itself up, and scramble out of the apparatus, Can sean 
any question that the animal would have learned to do this? Anyone ene 
has attempted to train rats to avoid in the Skinner box and who has He 
taken the precaution to fasten the lid of the box securely can testify ve Bs 
ingenuity and persistence with which the rat will try to escape from the D Id 
after it has once succeeded in doing so. The Point is that standing up wou 


k £ 4 » k sgsful 
undoubtedly have been rapidly acquired if this response had been success 
in getting out of the apparatus, 


The point is nicely illustrated by un 
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laboratory by Perry Duncan, Rats were Put in a box about twenty inch 
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if it can perceive a way out, and it will freeze if it cannot. Two features of 
this analysis make it cognitive; one is that it emphasizes the perception-like 
factors governing behavior, and the second is that it emphasizes that it is 
these factors, rather than behavior per se, that are learned in the situation. 
The change in behavior is merely an index, a symptom, or a manifestation 
of what is learned. 

There is still one last way out for the traditional S-R associationist. It 
could be argued that even though the frightened rat has a limited reper- 
toire of defensive behaviors, learning can still be assumed to be based upon 
the strength of S-R associations if we include the operation of a punishment 


mechanism. The argument is basically that if the rat can only flee or freeze, 


then the real impact of avoidance training is the punishment that it pro- 


vides for inappropriate behavior. If we require the animal to run in the 
shuttle box, then it is necessary only to punish freezing, and this is accom- 
plished by programming the avoidance of shock to be contingent upon 
1 series of trials freezing will be weakened by the punishment 
g will emerge as the stronger response.! One 
Way to test this hypothesis is to try to establish freezing as an avoidance 
response by punishing attempts at flight. The final avoidance-learning 
experiment to be described here (Bolles & Riley, 1973) was an attempt to 

groups will be described. For one group 


do this, The results from four 
brief shocks were scheduled to occur every five seconds as long as the animal 
remained in motion. As soon as the animal began to freeze the series of 


shocks was terminated, and no more shocks were received until fifteen 
seconds after freezing had been discontinued. If freezing was resumed during 
this fifteen-second interval shock was again avoided. Thus, by continued 
freezing, or at least by not being active for more thag fifteen seconds ata 
time, the rat could avoid all shock. We may call this group the contingent- 
avoiđance group” because the avoidance of shock was contingent upon 
freezing. In the second group, which we may call the contingent-punish- 
ment group,” freezing was punished by a series of brief shocks every five 
seconds if it persisted longer than fifteen seconds. Thus, these animals could 
avoid all shocks by being constantly active or at least by engaging in some 
activity every fifteen seconds. One complication in the procedure was that 
the contingent-punishment group first received five minutes of avoidance 
training before being shifted to the punishment contingency. The learning 
curves for these first two groups are shown in Figure 10.2. 

Note the extremely rapid acquisition of freezing by both groups in the 


running. Over 
that it receives, and so runnin 


1 This solution to the avoidance-learning problem had been suggested some years ago 
by Sidman (1953) and by Kamin (1956) but had not appeared very plausible because it 
put a large burden upon the punishment mechanisms. It was revived later (Bolles, 1970) 
because the idea of a limited SSDR repertoire and a small number of alternative responses 


made it seem somewhat more plausible. 
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group,” received shocks scheduled randomly in time, averaging one every 
43 seconds, to match the shock density of the contingent-punishment group. 
The performances of these animals are also shown in Figure 10.2. 

hing. The punishment contingency appears 
to have no effect; exactly the same suppression of freezing was found when 
we simply shocked animals randomly in time as when we explicitly pun- 
ished freezing. There still appears to be a small but genuine avoidance 
effect, but it too appears to be an artifact. Consider the following observa- 


tion. Under the conditions that we used, that is, with the particular shock 


intensity, the particular apparatus, and so on, we found that shock invari- 


ably produced a variety of violent reactions. It always broke up freezing. 
Then, in about ten seconds, on the average, the rats would begin to resume 
he probability of freezing would increase toward 100 percent. 
ous shock the probability of freezing always 
rose to 100 percent and stayed at that level for a number of minutes (see 
Figure 10.3). As the animals gradually began to move about and the prob- 
ability of freezing gradually decreased, the durations of movement became 


longer, so that after fifteen minutes (900 seconds, on the average) a con- 
secutive period of fifteen seconds of movement occurred, defining the avoid- 
ance contingency. At this point the contingent-avoidance animals | 
another shock. What was really happening to animals given avoidance 
training was that, following a shock, they would move for about ten seconds, 
would then freeze for a number of minutes, and would break off freezing 
for a moment, accumulating a few more seconds of nonfreezing time. As 
soon as fifteen seconds of movement had occurred the animals would be 
recycled back to the start of t 


he freezing function (Figure 10.3). In other 
words, the apparent learning curve 
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Fig. 10.3 How the probability of freezing in the 
experimental situation is assumed to change with 
time following the last shock. 
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differences do not average out. The 


ung but freezing to the group average, but aniraa 
reduce the group average ada 
analysis of the variance among animals shows that thi 
almost exactly for the difference 1 
Other aspects of the d 
accepted sense of the word. The perform 
dropped immediately as soon as the punishn 


so that there was no “learning curve” under the contingent-punishment ieee 
dition. As soon as the punishment contingency was introduced the rate O 
freezing dropped immediately to the new leve] appropriate to the rate i 
which shock was being received, which was dictated entirely by the rising 
portion of the freezing function diagrammed in Figure 10.3. A final indi- 
cation that there was no learning in any group is that the time following 
the last shock at which the animals resumed freezing averaged approxi- 


mately 10 seconds for all groups. That is, animals that were punished for 
freezing resumed freezing just as quickly 


: as animals that avoided shock 
by freezing. The programmed consequences of freezing had no effect what- 
ever upon the latency of animals’ resumptio 


anim: n of freezing. Bolles and Riley 
therefore concluded that in Spite of the fact j 


: that freezing can be controlled, 
at least to some -xtent, by making shock either 
or the nonoccurrence of 


i 5 contingent on the occurrence 
Teezin: P 
i 8 3 trol cannot be said to be 
produced by the experimental contingencies per se, because it an be ea 
a unted for in terms of the fi i y a a eS 
pletely acco unction n in Figure 10.3. Freez 
ie r avoidance and punish- 
itions; this difference accounts f, : 
ment condition: 1} or the ifferent results so com- 
pletely that there is no reason to suppose that any animal’s beh 
being affected either by punishment or by any other Programy 
quence of its behavior. 


atistical artifact. 


artifact accounts etween groups. A 
ata suggest an absence of learning, in the genera . 
ance of the punished anima 

nent conditions were introduced, 
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We have the paradox of what appears to be very effective avoidance 
completely described in terms of the con- 
ditions that elicit it. In a sense, Figure 10.3 describes the entire experiment 
and tells us everything we might want to know about freezing. Let us con- 
sider, then, the nature of this all-important freezing function. The absence 
of freezing in the first few seconds following shock is simple enough to 
explain: It was caused by unconditioned reactions to shock. The animal was 
leaping about, squealing, and running. All these responses are what we 
ordinarily think of as elicited, or respondent, behavior. The interesting 
question “iS why we obtained prolonged bouts of freezing lasting fifteen 
minutes on the average. Are they due to a persistent emotional reaction? 
Are they caused by a high level of arousal? Further unpublished research 
from our laboratory indicates that none of these possibilities accounts for 
the effect, except in small part. If an animal is removed from the situation 
a few seconds after it is shocked and placed in a different box, very little 
freezing is found. It is clear, then, that this freezing occurs simply because 
the animal remains in the situation in which it was shocked. The simplest 
way to describe it, and perhaps the only way to describe it, is to say that the 
rat continues to freeze in the shock situation for several minutes because it 
expects shock there. When the rat is placed in a neutral situation where 
o shock, there is no expectation of shock, there is very little 
, and there is very little freezing. 
ather involved analysis that the rats’ freez- 
is dictated almost entirely by the char- 
ion. If the situation is perceived as inescapable, which 
1 will continue to do so, on the average, 


learning, yet the behavior can be 


there has been n 
residual emotional reactivity 

We may conclude from this Š 
ing behavior in the shock situation 
acteristics of the situat 


of course it is, the rat will freeze anc n 
for fifteen minutes. We suggest that at that point the expectation of shock 


weakens and is replaced by the expectation of safety. Freezing stops, and 
normal patterns of activity resume. There is one further conclusion that we 
can draw from this study: If we think of the animal as having just two 
classes of behavior, freezing and activity, then the fact that the one class, 
freezing, is not an operant implies that the other class is also not an operant. 
If the TAE could learn to run to avoid, then that learning would have been 
manifest in less freezing when freezing was punished. By the same token, 
ifit were possible to weaken running by punishment, then we should have 
seen a gradual learned increase in freezing. Our tentative overall conclusion 
from recent avoidance-learning studies in our laboratory is that changes in 
behavior that occur in an avoidance-learning experiment are caused entirely 
by the animal's altered perception of the situation, specifically its altered 
perception of what is dangerous and what is safe. There seems to be no 
reinforcement mechanism and no punishment mechanism operative in the 
avoidance-learning situation. We therefore suggest that the animal behaves 
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as it does entirely because it expects shock here 
Because of these expectancies it gives us SSDRs: 


tions, approach toward safe locations, 
ceived. 


and it expects safety there. 

flight from dangerous loca- 
fa safe i T- 

and freezing when no safety is pe 
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This account of avoidance beh 
expectancy theory. It may seem 
passed over during the 1950s 
the 1970s, but that is what seer 
are Not too committed to eith 
work are now turning to 
undoubtedly a number o 
Tolman’s origin 


, ; T ’s old 
avior draws heavily upon Tolman’s 0 
ironic that, after his 


tem was largely 
and 1960s, 


it should be rejuvenated during 
ms to be happening. Many psychologists who 
er the Skinnerian or the neo-Hullian frame- 
a much more Cognitive point of view, ‘There are 
f reasons for this current trend, but one is that 
al expectancy theory was never discredited; it was simply 
outstripped in popularity by alternative Points of view that seemed, for 
a time, more attractive. Partly because they were so attractive and generated 
so much research and theoretical] activity, we have begun to see the limita- 
tions and defects of these POints of view. A second reason for turning to 
cognitive models of animal learning is the great success that this type of 
approach has enjoyed in the field of human learning, Although many of 


the early learning theories were initially based on animal studies and then 
generalized to the human case, the tables seem to be turned, and in some 
respects current animal-learning theorists seem to |} 
psychologists studying human learning. Another re 


cognitive interpretations of animal behavior 
positions that have been dominant Over the | 
undermined by the work of Rescorla, K 
tioning and the work of Garci 
longer possible to conceive of the basic 
was possible just a few years ago. Cla 
different, and recent studies 


æ following the lead of 
ason for the trend toward 
at the strength of the 
ast few years has been seriously 


à, and others in classical condi- 
ers in oper 


is th 
amir 


ant conditioning. It is NO 
in the same manner that 
itioning now looks quite 
ave revealed that it 
8S, upon the species 
Ept the old view of 
accept reinforcement 
we are in desperate 
> and the cognitive approach 
nng and at the same time 


longer 
Operant learnin 
need of some kind of conceptual reorganiz 


suggests itself as relatively fresh an 
relatively familiar. 

Modern cognitive psychology 
earlier work. Many of the ideas 


8- In short, 
ation 
d unconfi 


obviously owes 
and muc 
his. The idea that learning consists of th 


a 
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e acquisiti 


8reat deal to Tolman’s 
anguage were originally 
on of information about 
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rather than the attachment of particular responses to 
owing to Tolman. He defended it at a time 
when almost no one else did.2 But modern cognitive psychology is not just 
a revival of Tolman’s position, and it is surely not a step backward over the 
last forty years. Let us note some of the important advances that have been 
made during these years. There is the obvious fact that we know enormously 
more about animal learning and motivation than we did then. There is 
the further advantage that we have a much more sophisticated technology. 
We have better equipment, better experimental designs, and better experi- 
mental traditions and methodologies. There have also been some important 
conceptual advances It seems undeniable that much of Tolman’s insight 
into animal behavior was derived by analogy from his understanding of the 
human mind. Tolman probably saw animal learning as a rough facsimile 
of the acquisition of knowledge by man. Contemporary cognitive psy- 

the analogy of the computer. The 


chology is based much more upon 
heuristic model is not the mind of man but the computer. We refer not to 
knowledge, ideas and perceptions so much as to input and assimilation, 


as well as to the storage and retrieval of information. We talk about the 
ather than about what the subject knows. 
advantage that we enjoy today in developing 
know how to oper tionalize our cognitive 
yerimental procedures. We have a number of extremely 
al procedures and paradigms that are well worth retaining 
once we translate them into informational terms. For example, we may 
conceive of a classical-conditioning experiment as a situation in which we 
as experimenters introduce a certain lawfulness or orderliness into the 
animal's world. One event s* always follows a prior event S. With this 
procedure we are conveying, Or attempting to convey, to the animal some 
information about its world. If the animal's behavior changes, we may say 
that it has assimilated this information, at least in part. 
The cognitive psychologist certainly does not have to be a mentalist, 
though he may be one if he is so inclined. He may speculate about the 
awareness or the perceptions of the animal, as T have: done here to empha- 
size a point, or he may use a more easily operationalized language, the “as 
if” language of Tolman, in which critical terms such as expectancy are 
given operational definitions. But he also has the ppugn of uang a strictly 
objective language; he may describe the information embedded in or con- 


veyed by his experimental situation and merely record how the animal's 
ormation 1S presented. For example, he may 


the environment, 
particular stimuli, is certainly 


processing of information © 
But perhaps the greatest 
a cognitive theory is that we now 


concepts into ex} 
useful experiment 


behavior changes when this inf 


this; the Gestalt psychologists were actively pressing 
Itists had considerable impact on psychology gen- 
e on the mainstream of learning theory. 


2 Tolman was not entirely alone in 
the same point. But although the Gesta! 
erally, they had relatively little influenci 
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describe a Pavlovian experiment as one in which a particular R 
S+, predicts S*, and another stimulus, S—, predicts no S*. This ee 
is cognitive, but it is language about the procedure, not about psycho F ae 
n cognitive psychologist thus really has two kinds 


i ructs like 
aini y etic: astructs li 

guage containing hypothetical cor 

ess certain st 

ining 


expectancy and words of | 
procedural language conta 
memory that can be appli 
tioning experiments. He ¢ 
describe these experim 

We may note th 


atus like perception, as well as : 
terms like prediction, information, ae 
ed to operant-conditioning and een 
an use an information-processing language t 
ents. a 
Oretical positions, as typified by the early 
theorists discussed in this book, not only have different conceptions of the 
nature of the learning process; they must also necessarily have different 
conceptions of the learni r example, for Hull reinforcing 
a rat in a Skinner box meant producing the learning of new responses. For 
Skinner it means controlling the animal's rate of responding, something 
quite different. For Tolman it meant satisfying an animal's demand oo 
confirming its expectancy, Something different again. So the new cognitive 
psychology not only gives us a partly new and partly old idea of what the 
learning process Consists of; it also gives us what may be much more im- 
portant, a new look at Our familiar e tal procedures, Having once 


as a procedure for making an 
: : > We can no longer see it as just a procedure 
for connecting a CR to a CS, 
Not all modern cognitive k alike, of course. There is a 
all pay homage to Tolman 
» Dut he is not our leader or 
r efforts in different directions, emphasize 
4 Tk on different kinds of problems, and, of 
t ideas about just wh 


i . at cognitive psychology is- 
Rather than attempting to survey this diffuse and 8 psy 8y 
the remainder of this chapter wil] develo 


processing language in which the key term 
onceptualizing what I believe ; E 

term for concep ae al ei most importan 

aspect of many learning Situations. In the y 

briefly a variety of different learni 


is is my own 
t procedural 


. k S chapter we will survey 
ng situations P 


eens to show ho an be 
interpreted in terms of predictive cues, w they ca 
Predictive Cues 
It has already been noted at several points ip Precedin, 
the classical conditioning situation is one i 


$ 8 discussions that 
stp ^ fixed S.ga relationship 
p eTYthing else is 


is imposed upon the animal. In a Pavlovi riment S 
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held as constant as possible so that, ideally, S and S* are the only events 
conveying any information to the animal. The S is then a predictive cue for 
S*. Pavlov said it first: The S “signals” $*. The recent work of Rescorla 
described in Chapter 8 indicates that what the animal learns in a condi- 
tioning experiment can be aptly described as the predictive relationship 
between S and $*. Rescorla has also shown that, if the predictive relation 
is reversed so that S* is more probable in the absence of S than in its 
presence, then we obtain inhibitory conditioning. The inhibition ofa 
response that would otherwise occur can be found with either fear (Rescorla) 
or salivation (Pavlov). Again we clearly have a predictive relationship 
between S and S$*; it is simply reversed. S predicts no shock or no food. 
It is convenient to designate the predictor of S* by the symbol SE, and a 
cue that predicts the absence of S* by S—. The establishment of conditioned 
inhibition therefore can be regarded simply as an experimental procedure 
in which S— predicts the absence of S*, whereas the context or some other 
S+ predicts the occurrence of S*. Discrimination training can be regarded 
as a procedure in which two kinds of predictive relations involving both an 
S+ and an S— are imposed on the animal more or less simultaneously. 

There is one complication: The S is always presented against some back- 


ground of stimuli; there has to be an apparatus, and there have to be 
and they too are part of the total informa- 


revaili i s conditions, 4 
ar aerate observed, even when a dog has peen well trained 
to salivate to a bell in the laboratory, it is not likely to salivate to the bell 
out on the street. It is a well-known observation from fear-conditioning 
studies that a rat that shows suppression to a tone predicting shock in the 
Skinner box may show little suppression when the tone is presented in a 
different experimental situation. Unfortunately, we know relatively little 
about the extent to which a cue conveys information apart from the experi- 
mental episode in which the cue is embedded, but it is possible to conceive 
of a variety of experiments to answer this question. To what extent does 
the context itself become a predictor? To what extent do different kinds 
of contexts have different predictive values? Is it possible, for example, that 
a rat will more readily learn to avoid a place where shock has been ex- 
perienced than it will learn to react in some other way to a light or a 
tone that provides a contextual cue? We know from a variety of research 
on the Garcia effect that rats are more likely to avoid food with a distinc- 
tive taste that has been associated with illness than they are to avoid food 
with a distinctive odor or food in a distinctive location. We know further 
that other species of animals may show different hierarchies of cue utiliza- 
tion. 

When behavior changes during a conditioning experiment, we assume 
that the change reflects the learning of the predictive relationship. We 
assume that the predictive relationship itself becomes internally represented 
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as an expectancy. The basic theoretical question then becomes ae 
animal behaves in a particular way when a particular S* is predicted. ae 
did Pavlov's dog salivate when the bell was sounded? There are two possib ; 
answers to this question. One requires falling back on the procedural is 
Operational aspect of the predictive cue: The bell is a predictor of food. 


i i i E ; isfactory because it 

This answer is Operational, but it js not entirely satisfactory becau Ns 
- : ; s ; sheth 

bypasses the question of learning. The bell is a predictor of food whet 

or not the dog has learned 


anything 
usually has to undergo a m 


: ip. The animal 
about the relationship. The anit 
umber 


of trials before the CR occurs. So Ine 
cognitive psychologist must give a second answer, which is that the dog 
salivates because it expects food. Note that this answer is ultimately a tautol- 
ogy, but then so is any answer to the question of what is learned. The 
cognitive answer is no more tautological than the 
the dog salivates because the Salivary 


the CS. The one answer is just 


it is simply a different answer based upon a differe 

the results of learning experiments are to be explained, 
The predictive-cue analysis of instrument 

more complex because more kinds of infi are conveyed by the 

different possible procedures, Typically, however, the operant situation 

contains an SP, a stimulus j 

This stimulus becomes ; 


; w that 
conventional answer thé 
iti to 
response has become conditioned 
as basic anq Just as explanatory as the other; 


i w 
nt conception of ho 


mna little 
al or Operant learning is a litt 


ormation 


hich responding is reinforce 

€ reinforcer, the $*. The pi 
al in th 
same token, the 


again there 
present. Thus the basic S-R-S* oper: 
tional predictive relationships, 
Consider a simple learni 
alley to obtain food. On the i 
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animal's own behavior becomes 
Ondition that gp must also be 
adigm contains a pair of condi- 


is the ¢ 
ant par 
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resent in the start area 
a few trials, particularly 
on 
foes ee expect food 
produce salivation, a general state of 

locomotion. Then as the g 


atus With 5, 


running at a high rate of speed. 

It should be apparent that the pheno 
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such a subtle and esoteric cue as the ], 
ment, the lapse of time since 


ately the animal will be 


menon that Ski 
which the 
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the last respon 
since the last reinforcement—oy it ani 


criminated thing like the color of th 
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Predictive cues perform the same kind of function rg and rg did in 
Hull’s secondary learning system. We have been presenting an analogy or a 
translation of what Hull, Spence, and other theorists in that tradition 
called incentive motivation. We noted in Chapter 8 that in the hands of 
tivation was an extremely powerful explanatory 
rer (1960) has suggested that, given sufficient 


incentive motivation, a response did not have to have its own habit; a 
reinforcement mechanism was unnecessary. Mowrer's argument can be 
translated into the present framework if we propose that predictive cues 
constitute a sufficient explanation of sior; a reinforcement 
mechanism is unnecessary. For many of his readers, however, Mowrer’s in- 
centive-motivation theory had an unpleasant feel of incompleteness about 
it, and perhaps so does this simple account of predictive cues. Some addi- 
tional mechanisms can be introduced to fill it out. 

echanism simply recognizes that animals do not respond 
dictive cues but respond to them in particular 
mal looks down the alley, it can see portions 
ore proximate predictors of food than 
that when there are good predictors 


some theorists incentive mo 
mechanism. For example, Mow 


an animal's bena 


One possible m 
diffusely and at random to pre 


ways. For example, as the ani 
of the apparatus that are better or m 
the cues where it is. We may assume 
at will prefer the good ones and that, when there 
listal predictors, the rat will approach the 


anism that ensures that the animal 


and poor predictors, the r 
are proximate predictors and ¢ 


proximate ones. We thus have a mecha 
will run down the alley, rather than Just salivating or engaging in diffuse 


excited activity. The importance of this kind of mechanism has been em- 
phasized by Bindra (1974). According to Bindra's cognitive interpretation 
of behavior, the stimuli lying just ahead of the rat in the alley are particu- 
larly good predictors of food; stimuli arising in the goal box are still better; 
those arising from the food cup are the best. So there pa gradient of pre- 
dictiveness that produces a gradient of approach behavior. The same angu- 
ment can be applied to the Skinner box. The box as a whole is a contextual 
cue, a diffuse and distal predictor of food. The discriminative stimulus is a 
better predictor. The conjunction of these cues with the stimuli arising 
from the bar itself is a still better predictor. Therefore the animal ap- 
proaches the lever and manipulates it. 
There is the further consideration that once the bar becomes a predictor 
great deal the rat can do with it besides manipulate 
it. (It is probably not just a coincidence that the, pasture and the paw 
in bar pressing are topographically similar to those 

matory response.) So the occurrence of operant 
the peculiar stimulus properties of the 


of food, there is not a 


movements involved 
accompanying the consumi 


bar presses is explained partly by 
bar itself and partly by the fact that the bar is a predictor of food. Bindra 


(1974) proposes that in other situations, in which there are other objects to 
manipulate and different paths to navigate, these two mechanisms—the 
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predictiveness of the cue and its speci 


TA a -ties—can 
al response-eliciting properties—c 
often be counted on to zero the anim 


al in on the appropriate response. 

Possible mechanism, The operant situation is conven 
of a response contingency and not by the stimulus 
Properties of the situation, Perhaps the animal learns specifically that 


running in the alley produces food and that pressing the bar in the Skinner 
box produces food. Perhaps the 


i -i a a 
animal learns a specific expectancy of 
form that Matches the definir 


18 contingency, Perhaps rats are capable 
learning not only expectancies of the S-S* form but also Sh peclanees W 
the R-S* form, Such expectancies were hypothesized by Tolman (he called 
them “means-enq expectancies”), and they have been adopted by such recent 


writers as Irwin (1971) and Seligman and Johnston (1973). This general 
conception y 


paradigm into the cognitive S ember that the operant 
a reinforcement contingency 1 
» as we have seen, con- 
constraints means that there 
GA kinds of information that an animal can process 
and utilize, Can the rat real] Xpect that its behavior will have 
certain consequences? And, if it can, then why do we find constraints on 
aa z A ar 

learning? Why did the Bre imals misbehaye? If the rat can learn 
that shock will not occur if jt makes a certain response, as Seligman and 
Johnston Propose, then why does the rat Perform so Poorly in a variety of 
different avoidance-learnj i ions? 

At the present time there 
there is clearly no consensus 

97 av 5 

where (Bolles, 1972) I h Omise Position that involves 
two assumptions. The first is that S-S* sare ] 1 relatively 

. ar a a reli 
quickly and that they are generally rath Sars 


er accurate repr ati f S-S* 
5 aige ye a resentations o 
contingencies in the enviro second 


- oS “lower, more: wii 

bane, “ ae s innate response tenden- 
ight > : 

8 i Carn the relationship 

‘ponding expectancy, 

other anim i ; between bar pressing 

2 (aie capable of such learning 

OF the rat Suggests that such 

l or for 


: : Some reas be 
retrieved at the appropriate time. on cannot 


There is still a fourth possible mechanism ‘ : 
rant behavior. It is conceiy. bl “t might Operate to help 
control operant behavior, able that all oe tp 
type but that one of the important Predictive Cues is the rh: of the - 
consequences of the animal's behavior—perhan, Bie iate stimulus 


prioceptive feed- 
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back from responding. For example, it may be that the “feel” of pressing 


the bar is the best predictor of food. Then, when the rat has learned the 


appropriate expectancy, that the feel of pressing the lever precedes food, 


this expectancy produces the appropriate behavior through some type of 
retrieval mechanism. Back at the very beginning of American psychology 
William James (1890) observed that the proper execution of a skilled per- 
al to learn what the correct performance 
feels like. A tennis player placing a volley has to know what the necessary 
stroke feels like; if it feels right, then it probably will be right. Once that 
“feel” is learned, the response takes care of itself. Perhaps such a mechanism 
works for the rat, at least in some instances. 

It should be apparent that the cognitive theorist has a variety of mecha- 
nisms to explain what happens when learning occurs with the operant 
paradigm. He has a variety of alternative models, some of which are based 
exclusively upon the notion of predictive cues and some of which require 


formance requires the individu 


additional mechanisms. : 
The basic issue for the cognitive theorist 1s what kinds of information 
animal? We know that a predictive cue 


can be processed by a particular : 
also that a relatively specific S* will 


conveys not only that S* will occur but 
occur, Crespi's experiment, discussed earlier, shows that the rat not only 


expects food but also expects a particular amount of food. Such studies, as 
well as Tinklepaugh’s (1928) classic experiment indicate that an animal may 
expect a particular quality of food. There is a variety of evidence (sum- 
to indicate that, with proper training, rats can 


marized by Capaldi, 1970) : 
f reinforced and nonreinforced trials. 


come to expect particular sequences O l 
In the present section, however, we shall focus upon just one dimension of 


information processing: whether behavior is governed just by predictive 
cues or if it is also controlled at least partly by the information conveyed 
by the response-reinforcement contingency. Can animals acquire something 


like an R-S* expectancy? We can expect no general answer to this question; 
erent answers for different species of animals. There 


we must anticipate diff ; ] 
can be no question that humans can learn specific relationships between 
their behavior and the consequences of their behavior (for example, Estes, 

he interesting phenomenon of learned helpless- 


1969), With dogs there is the in 
ness, Seligman and Maier (1967) have shown that after dogs are given a 
series of inescapable shocks they perform very poorly on even the simplest 


aversive-learning tasks. The results from a number of such experiments 
suggest that during the preliminary inescapable-shock phase of the experi- 
ment the dog learns that there is no relationship between its behavior and 
the consequences. It learns that it is helpless, and the “attitude” of help- 
lessness transfers to the test situation to prevent any learning. The other 
side of the coin is that, in the absence of this prior traumatic experience, 
the dog must have something like a hypothesis about the solvability of the 
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problem. Presumably 
“learned confidence.” 

The rat seems to have a 
information about the conseq 


P . rs iv 1 
it would also be possible to train dogs to give then 


marginal ability. It appears able to process 
uences of its behavior in some situations but 
not in others. Rats appear much more able to learn how their behavior 
relates to food than how their behavior relates to a traumatic $* like shock. 
So we must suppose that, in addition to the existence of species specificities, 
there are also motivational specificities in the extent to which the response 
contingency contributes to behavior, The rat may simply be able to process 
more different kinds of information about how to obtain food than it can 
about how to avoid shock. 

In animals below the rat in information-processing ability, there seem to 
be obvious and serious restraints upon the kind of R-S* information that 


can be handled, Perhaps that is why the pigeon shows such a limited reper- 
toire of learnable responses, even th 


CRP EE Cen ough it has tremendous ability to learn 
sensory discriminations, Perhaps that is why the Pigeon is so easily duped 
Autoshaping, and more specifically the 
automaintenance, illustrate clear failures of pigeons 
anioni, The latter procedure pits the response-pro- 
tnp an S-S expectancy against what sh 
inhibiting effect of a negative R.-g* Expectancy, 
the S-S*- factor predominates almost totall 
jartz 3 Jillia 979) in 4: 
Schwartz and Williams (1972) indicates that the response contingency can 
have subtle but measurable effects on the pigeon’s behavior. There is also a 
variety of evidence showing that other animals May not be subject to auto- 
shaping. Gamzu and Schwam (1974) found little of it j ü ae 
° . : n 1e 
J ji Was Sex ks 
1 niversity of W ashington t in rats, and Jenkins (personal 
communication) finds little autoshapj i ; ibl lanation 
: . . SS 4 ani 
of these species differences is th there js ges ‘i SAR typed 
. Sa stereoly 

response that is used both for obtaining food and for ae Y i PAT 
3 ay, ror], , : s 4 aun it; W. — 
mammals have a variety of food-getting behaviors that are di ti 8 Ka ent 
from their consummatory responses, But a simpler į dis inctly c if oe 

different animals have markedly differe nterpretation is se 
t processing and using 


in the autoshaping experiment, 
phenomenon of negative 
to process R-S* inform 
ducing effect of ould be the response- 
In the case of the pigeon, 
Y, though recent evidence from 


Nt capacitia, 

R-S* information. Capacities fo 

If an animal cannot use R-S* 
the R-S* contingency can have 
may still be possible for the reinforcement co: 
indirect effect because the reinforcement Še 
cues that control the behavior through S-g» 
what happens, I believe, with the rat in a 
is electric shock. Recall the earlier discussion 
ment on freezing where it was found that th 
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high levels of freezing but that the data showed 
that this behavior was due entirely to S-S* mechanisms. That experiment 
also showed no weakening of behavior that could be attributed to the direct 
effect of the punishment contingency. It has now become apparent that the 
rat is peculiarly insensitive to R-S* contingencies in the punishment situ- 


ation. Let us look briefly at some rele 


very effective in producing 


vant research. 
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eriment reported by Azrin (1956). Azrin first 
food on a VI 3-minute schedule. Super- 
a discriminative punishment procedure. 
blue or orange light during alter- 
always safe; shocks were 


Consider the following exp 
trained pigeons to peck a key for 
imposed on this base-line task was 
The food key was illuminated by either 
nate two-minute periods. The blue condition was 
never presented while the key was blue. But when the key was orange shocks 
ording to one of four sets of experimental conditions. 
as contingent upon a key peck after a fixed time 
that is, the first response occurring after 


the light had been on for one minute was punished. This condition was 
the so-called fixed-contingent condition. There was also a fixed noncon- 
tingent condition in which the animal was shocked one minute after the 
onset of the orange light whether or not it had pecked the key. In the 
variable-contingent condition, the animal was punished for the first key 
peck at some randomly selectec (averaging one minute) after the onset 
of the orange light. Finally, under the variable-noncontingent condition, 

p acir behavior at unpredictable 


the birds were shocked independently of ul pal 
r the beginning of the orange 


times, which also averaged one minute afte 
light. Under each cond s thus set up, on the average, once 


lition shock wa 

during each orange key illumination. we : ; é f 

The different experimental conditions are a litue difficult to keep in mind, 
and they were evidently a little difficult for the birds to discriminate too; 
5 hat it required extensive training to reach asymptotic be- 
behavior had stabilized, it showed a very interesting 
grammed in Figure 10.4. Each experimental con- 
dition produced its own pattern of suppression. Notice that behavior oc- 
curred at the high rate characteristic of a VLS schedule in the presence of 
the safe blue light- But notice that suppression occurred at different points 
during the orange light as a function of the different experimental condi- 
tions. The results can be described objectively in a number of ways, but all 


such descriptions a” 
summarize the resu 
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Under one condition shock w 
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1 time 
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Fig. 10.4 Composite cumulative records of pigeons 

pecking for food while receiving shock either con- 


tingently (C) or noncontingently (NC) scheduled 
either ata fixed time (F) or at varied times (V) into 
the orange light. (From Azrin, The Journal of Psy- 


chology, 1956, 42, 3-21 by permission of The Journal 
Press.) 
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This conclusion is also supported by recent, stil] unpublished oO 
our own laboratory. We used a technique originally described i E 
Wooten, and Matthews (1970), in which animals work M 
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a cue comes on for a few minutes to signal 
that bar pressing might be punished. Punishment was scheduled on a VI 
2-minute schedule, so that this danger cue was comparable to the orange 
key light in Azrin’s experiment. As Figure 10.5 indicates, within a few 
sessions the cue produced marked suppression (measured as the proportion 
of total responses occurring in the presence of the cue). In other words, a 
cue associated with punishment quickly suppressed behavior. The basic 
question then is whether these animals discriminated the punishment con- 
tingency per se and withheld responses for that reason or whether their be- 
havior was simply suppressed by the cue because the cue predicted shock. To 
get at this question group 2 was run, for which the cue signaled punishment, 
as it had for group 1, but additional, noncontingent shocks were programmed 
during the noncue periods, that is, between cue presentations, at the same 
rate that they were received during the cue. Under such conditions the cue 
provides no information about the occurrence of shock because shock is 
arranged to be as probable in the absence of the cue as in its presence. The 
only information that the cue conveyed in this instance was that the punish- 
ment contingency was in effect. As Figure 10.5 indicates, this information 


line task and every few minutes 


Proportion of total responses 


= 4 a Le 
o 21-30 31-40 41-50 51-60 61-70 71-80 81-90 


11-20 
19 Sessions 


10.5 Proportion of total responding in the 
presence of a cue signaling for both groups that re- 
sponding might be punished. The difference be- 
tween the groups was that noncontingent shocks also 
occurred in the absence of the cue for group 2. 


Fig. 
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was evidently not used at all during the first thirty training sessions but was 
eventually reflected in a change in behavior. The initial results are com- 
parable to those reported by Rescorla (1968) showing that if a cue signals 
no increase in the probability of S*, then it has little effect upon behavior. 
What these results show, in addition, is that the specific information 
the response contingency is only very gradually assimilated. After ninety 
training sessions the discrimination of the contingency was still not com- 
plete. We are therefore led to the conclusion that in those cases where pun- 
ishment has an immediate suppressive effect upon behavior, it can be at- 
tributed entirely to S-S* learning and that, as in the two experiments just 
described, it is only when extensive training is given that the rat or the 


pigeon gradually begins to assimilate and use information about the response 
contingency. 


about 
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inability to put forth a convincing interpretation of how secondary rein- 
forcers are established. Hull and his followers assumed that secondary 
reinforcers are established by the Pavlovian operation of pairing an initially 
neutral cue with reinforcement. Thus, the box in which the rat eats becomes 
should then be able to reinforce the learning 
of new behavior even when food is withheld. The difficulty is that when 
food is withheld the animal quickly detects the worsening of experimental 
conditions and refuses to run. At any rate, after having examined recent 
developments in the interpretation of classical conditioning, we must be 
skeptical about whether pairing per se is the appropriate procedure for 
establishing secondary reinforcers. Perhaps it is the predictive value of the 
cue that is important rather than being paired with the reinforcer. ; 

We saw in Chapter 7 that as early as 1936 Skinner had started z5 nadi 
tion among operant conditioners of treating the secondary sentonm j or ri 
response as an S? for another response. The click in the Skinner box or i he 
color of the goal box in the runway situation 1S a secondary reinforcer 


because it has stimulus control over some subsequent behavior. But again, 
assical conditioning, we must wonder whether 


thing but the predictiveness of the cue. What 
is not a predictor of some upcoming sa 
reinforcer is fundamentally a source of 
ement was tested by Egger and Miller 
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supported bar pressing for the second group, or which it was informa- 


tive. A variety of controls and the results from another study (Egger & 
. é y to the general proposition that 


Miller, 1963) gave additional support : raie. $ 
the enectiveness Ob a secondary reinforcer in maintaining behavior de- 
if edictor of primary reinforcement. 
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why it is difficult to produce learning of a new response based on secondary 
reinforcement alone, that is, a cue unsupported by primary reinforcement, 
is that in this situation the secondary reinforcer is no longer a predictor of 
S*, and the animal very quickly makes the discrimination between the train: 
ing situation in which the cue predicts S* and the test situation in which it 
does not. It appears, then, that what controls the animal's behavior is not 
the reinforcing property of the cue per se but rather the value of the infor- 
mation about S*, 


There are some troubles and difficulties with this inform 


ation hypothesis 
of secondary reinforcement: they can be 


cleared up only by further investi- 
€ concept appears as profitable and 
reinforcement 
aises the interes 


gation, but in principle the predictive-cu 
instructive when applied to secondary 
to other learning phenomena. It r 
of whether 


as it is when applied 
ting question, however, 
a new concept of reinforcement may be salvaged from what 
remains of the old concept. If we think of s* as an event that is of value 
to the animal, one for which it will work or for which it has a demand, 
then S* is, for all practical purposes, a reinforcer. What Egger and Miller 
seem to have shown is that, just as S* is of value to the animal, so is infor- 
mation about S*, Just as the animal will work for S* and use predictive 
cues to obtain $*, so it will work for these predictive cues, 
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a toy, it will take some time for us to tire of playing with it. It seems safe 
to conclude that the current trend of growing interest in cognitive psy- 
chology is going to continue at least through the 1970s. 

In Chapter 9 we emphasized another current trend, the attempt to relate 
learning theory to certain broad principles of the biological sciences, particu- 
larly the principles of evolution. Specifically, we have begun to appreciate 
the fact that an animal's behavior is as much a reflection of evolutionary 
selection pressures as are its form, coloration, and other physical charac- 
teristics, An animal's sensory capacities and its motor capacities are obvi- 
ously dependent upon its ecological niche, but it has not been so obvious— 
although it is becoming more so—that the ability to learn and the kinds of 
acterize a particular animal are also dependent 
upon how the animal has evolved under particular ecological pressures. 
Breland and Breland (1961) report that, whereas the dog or the rat may 
tend to become more active when it is deprived of food, an animal like the 
cow clearly does not. And the common cat, as it turns out, is a very poor 
subject in a Skinner box; one reason is that, instead of working for food 
and then going over to the food hopper. the cat becomes increasingly inclined 
to sit in front of the food source and wait. It does not seem to follow the 
laws of behavior that have been developed over the years to explain learning 
in laboratory animals; it simply behaves like a cat: a 

The discovery (Or recognition) of these species-ty pita behavior styles sug- 
gests that the laws of learning that have gradually emerged from the animal 

ve a rather narrow range of applicability. They apply, at best, 

and the pigeon, and their application to other animals must 
yeriments. On the other hand, we may anticipate 
that the study of learning in other animals will reveal other laws of learning 
that may well conflict with those that are already familiar. The prospects 
aging. The laws that we have been attempting to produce 
ttle generality, and the task of finding truly general laws 
to have been Just begun. But a more optimistic 
f learning that will ultimately be obtained 
q into a better and more meaningful framework than that 

based on the traditional assumption that it did not matter what animal we 
study because all animals are the same. And the task need not be so difficult 
either, because ROW that we have recognized the scope of the problem and 
A a broad biological principles to guide us, we may be able to find 

ut learning in other animals in much less time than the 
seventy years OF so that it has taken us to find out something about learning 
iñ the rat. In any case, if general laws of learning are ultimately found, they 
a much stronger claim to generality than the traditional laws that 
ierely assumed to be general. It seems safe to conclude that the 
f evolutionary considerations into learning theory is a develop- 
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ai hese 
ment that cannot be undone—and that the current emphasis upon t 
considerations will continue at 


least for the next several years 

In Chapter 8 we noted another general trend. In recent years there ta 
been a decline of interest in the major theoretical positions and an increased 
reliance upon a more procedural and more empirical approach to the 
phenomena of learning. Classical and Operant conditioning are most com- 
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irs we have witnessed the discovery of a variety 
of new phenomena such as autoshaping and the Garcia effect. Avoidance 
learning and punishment are not new phenomena by any means, but the 
widespread empirical and theoretical interest in them is new. Other inter- 
esting learning phenomena will surely be discovered, and the importance of 
phenomena that are now known but neglected will be revealed. This diver- 
sity of situations in which learning is now being studied is another healthy 

mena are understood the laws 


at when these phenor 
1 be embedded in a theory of greater 


heory will then have a broader 


Finally, in the last few yea 


development. It means th 
of learning that they reveal can ther 
generality. The ultimate encompassing t 


empirical base. 
In conclusion, we should emphasize that theory development proceeds on 
rusion of ideas and material 


Js. It proceeds by the int 
and comparative psychology. 


several different leve 
ology 


from such other areas as cognitive psych y 
analysis of its own methods; the theory of learning is 


procedures used to produce learning. 
analysis of particular phenomena. 


It proceeds by an 
greatly stimulated by analysis of the 


Theory also proceeds by the empirical 0 r 1 
If we really had a grasp of why behavior occurs in the way it does in the 


autoshaping situation or in the punishment situation, then we would be 
well on our way toward developing a theory that would encompass both of 
these phenomena as well as the more traditional phenomena that entered 
into learning theories. There are great challenges to anyone who would 
develop a new comprehensive theory of learning. The challenges are greater, 
the task is bigger, and the competition more able than when Watson or 
Guthrie advanced their theories. But I am confident that it will be done— 


perhaps by some enterprising reader. 
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